-- Logs begin at Wed 2022-06-15 23:39:29 UTC, end at Thu 2022-06-16 00:47:39 UTC. -- Jun 16 00:12:53.228725 dv-cinder-845990-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 00:12:53.237250 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 00:12:53 2022] *** Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: nodename: dv-cinder-845990-2-iscsi Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: machine: x86_64 Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: clock source: unix Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: pcre jit disabled Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: detected number of CPU cores: 4 Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: current working directory: / Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: detected binary path: /usr/bin/uwsgi-core Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: your processes number limit is 47680 Jun 16 00:12:53.239251 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: your memory page size is 4096 bytes Jun 16 00:12:53.239875 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: detected max file descriptor number: 2048 Jun 16 00:12:53.239875 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: lock engine: pthread robust mutexes Jun 16 00:12:53.239875 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: thunder lock: enabled Jun 16 00:12:53.239875 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 00:12:53.239875 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 00:12:53.283751 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: Python main interpreter initialized at 0x5648074cf9c0 Jun 16 00:12:53.283751 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: python threads support enabled Jun 16 00:12:53.283751 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: your server socket listen backlog is limited to 100 connections Jun 16 00:12:53.283751 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: your mercy for graceful operations on workers is 90 seconds Jun 16 00:12:53.284110 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 00:12:53.284186 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: *** Operational MODE: preforking *** Jun 16 00:12:53.284427 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: *** uWSGI is running in multiple interpreter mode *** Jun 16 00:12:53.284427 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: spawned uWSGI master process (pid: 86539) Jun 16 00:12:53.284630 dv-cinder-845990-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 00:12:53.285323 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: spawned uWSGI worker 1 (pid: 86548, cores: 1) Jun 16 00:12:53.285542 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: spawned uWSGI worker 2 (pid: 86549, cores: 1) Jun 16 00:12:53.286555 dv-cinder-845990-2-iscsi devstack@keystone.service[86539]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 00:12:58.846755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:12:58.846755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: from cryptography.utils import int_from_bytes Jun 16 00:12:58.847532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:12:58.847532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: from cryptography.utils import int_from_bytes Jun 16 00:12:58.853291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:12:58.853291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: from cryptography.utils import int_from_bytes Jun 16 00:12:58.853838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 00:12:58.853838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: from cryptography.utils import int_from_bytes Jun 16 00:12:58.898586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 00:12:58.899067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 00:12:58.899269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] command line args: [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 00:12:58.899537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 00:12:58.899714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ================================================================================ {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 00:12:58.899968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] admin_token = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.900209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] config_dir = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.900393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.900573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] config_source = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.900755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] debug = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.900980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.901343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.901521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.901771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_config_append = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_dir = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.902965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_file = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.903179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.903359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 00:12:58.903493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.903569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 00:12:58.903683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.903756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] command line args: [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 00:12:58.903893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.903983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 00:12:58.904060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ================================================================================ {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 00:12:58.904250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] admin_token = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] config_dir = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.904857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] config_source = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] debug = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.905913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_config_append = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.906935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] publish_errors = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_dir = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_file = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.907962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] standard_threads = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.908991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] use_journal = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] use_json = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] use_stderr = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] use_syslog = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.909978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.910148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.910224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.910386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.910461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.910645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.910787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.910868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.911118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.911193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.911336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] publish_errors = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.911413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.911571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.911652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.911804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.911888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.911984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.912060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.912230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.912306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.912466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.912554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.912703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] standard_threads = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.912779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.912974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.913221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.913304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.913575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] use_journal = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.913788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] use_json = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] use_stderr = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.913994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.914095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] use_syslog = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.914171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.914326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 00:12:58.914962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.915965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.916946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.917920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.918997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.919955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.920903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.921892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.922971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.923984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.924972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.925862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.926928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.927968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.928924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.929875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.930071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.931875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.932867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.933954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.934987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.935138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.935228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.935379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.935621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.935886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.936130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.936390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.936617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.936862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.937128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.937367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.937608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.937846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.938081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.938315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.938557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.938793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.939040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.939280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.939540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.939782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.940044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.940312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.940558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.940795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.external = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.941055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.941358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.941595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.941830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.password = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.942063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] auth.token = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.942308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:58.942550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.021811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.022040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.022304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.external = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.022564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.022894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.023149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.023398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.password = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.023645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] auth.token = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.024191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.024872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.024872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.025051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.025200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.025458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.025696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.025937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.026189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.026442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.026695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.026939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.027189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.027429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.027686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.027943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.028998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.029861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.030994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.031988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.032867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.033940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.034995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.035947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.036965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.037955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.038882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.039045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.039122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.039283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.039369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.040991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.041924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.042953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.043915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.044924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.045844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.046969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.047993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.048957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.049935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.050877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.051990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.052959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.053928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.054961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.055798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.056863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] role.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] role.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.057932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.058919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.059885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.060854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.061769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] role.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] role.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.062982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.063971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.064925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.065917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.066944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.067892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.068866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.069834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.070861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.connection = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.071888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.072841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.073980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.074944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.connection = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86549) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 00:12:59.075746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.075995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.076244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.076485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.076730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.076997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.077244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.077497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.077741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.077993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.078238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.078494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.078739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.078992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.079231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.079478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.079721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 00:12:59.079897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 00:12:59.105359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.105592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.105821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.109158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.109400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.109630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.114164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.118733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.142444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.142671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.144511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.144748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.198435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.201205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.248056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.248336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.251045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.251285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.316769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.317978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.318105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.318105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.318388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.318513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.341660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.341887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.342960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.343089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.343218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.343349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.343349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.353838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.354301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.366641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.366871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.367030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.367172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.367308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.367442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.368078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.368698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.368847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.369007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.369087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.369160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86549) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.369233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.369233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86548) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 00:12:59.386273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.386480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:12:59.386836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.386980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.387078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:12:59.387183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:12:59.387448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.387529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.387701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.387782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 00:12:59.387952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.388033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.388199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.388278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.388474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.388730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.399349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.399787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.400909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.408532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.409141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.411819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.412093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.412369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.412453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.412644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.412735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.412945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.413989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.414068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.414254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.414336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.414603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.415048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.415048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.415239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.441433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.442675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.443039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.443312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.444301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.444581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.448840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.449145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.449423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.449943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.450223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.450501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.456766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.457068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.457965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.458248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.462161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.462437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.462714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.462988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.463256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.463338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.463524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.463607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.463787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.463868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.464047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.464131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.464352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.464627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.464911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.465180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.483042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.483314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.483977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.484256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.487874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.488826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.491520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.491810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.493080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.493387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.497702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.498529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.501398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.502182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.505023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.505317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.505596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.505876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.505957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.506998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.507277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.527382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.527663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.527926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.528200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.529044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.529332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.529602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.529865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.536039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.537763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.537850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.538033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.539835 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.540971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.541343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.541622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.541909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.542197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.542470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.543101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.543393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.543675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.543964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.544242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.556281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.556553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.556829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.557134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.557410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.557680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.557956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.558533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.558814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.559096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.559380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.559662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.559939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.560216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.576386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.578309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.579617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.581607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.582915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.584497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.584934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.586332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.586797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.586875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.586949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.588375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.588651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.588950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.594206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.595817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.596259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.597927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.599063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.599358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.599640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.599913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.601197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.601489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.601776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.602799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.609810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.610103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.610378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.611798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.612083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.612365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.618404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86549) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.618676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.618947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.619212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.619477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.619764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.620032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.620305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.620456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86548) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 00:12:59.620672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.620764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.620961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.621856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.622019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86549) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.622100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.622321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.622696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.622971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.623250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.623526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.623793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.624055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86548) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 00:12:59.660353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5648074cf9c0 pid: 86549 (default app) Jun 16 00:12:59.662656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5648074cf9c0 pid: 86548 (default app) Jun 16 00:12:59.663744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5df18-ce72-41f6-8c50-4f9adbcad446 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:12:59.664041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5df18-ce72-41f6-8c50-4f9adbcad446 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:12:59.664322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5df18-ce72-41f6-8c50-4f9adbcad446 None None] PATH_INFO: `/v3` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:12:59.669390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Thu Jun 16 00:12:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:07.428215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:07.428784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:07.428784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:07.433845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:07.439005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:07.439452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:07.439571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:07.510851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: INFO dbcounter [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] Registered counter for database keystone Jun 16 00:13:07.531693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG oslo_db.sqlalchemy.engines [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86549) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 00:13:07.537770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writer thread running {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 00:13:07.715240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG passlib.handlers.bcrypt [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86549) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 00:13:07.715597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG passlib.handlers.bcrypt [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86549) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 00:13:07.770817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:07.772431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:07.895079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 00:13:07.927642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:07.998685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: INFO dbcounter [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] Registered counter for database keystone Jun 16 00:13:08.017235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG oslo_db.sqlalchemy.engines [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 00:13:08.023478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writer thread running {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 00:13:08.142522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b32c9-dc25-43a8-b483-f7e6e6cca7e6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:08.146714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:08.147376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:08.147666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:08.147949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:08.156225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:08.208058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 00:13:08.208400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33e93e-25a0-4bbd-9a49-1faf4a038118 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:08.218027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Jun 16 00:13:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:13:08.224011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:08.238150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0af028-bf0a-4dac-bae2-6caa49fe8d01 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:08.242968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:08.244133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:08.244424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:08.244702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] PATH_INFO: `/v3/projects/3c21334330db408fb0ba251a2f04ab82` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:08.246219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] RBAC: Authorizing `identity:get_project(project_id=3c21334330db408fb0ba251a2f04ab82)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:08.296320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING keystone.common.rbac_enforcer.enforcer [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 00:13:08.296616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f1930-8933-4d70-8d89-e8df8575e26c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:08.301166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Jun 16 00:13:08 2022] GET /identity/v3/projects/3c21334330db408fb0ba251a2f04ab82 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:10.438026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78b3c2-782f-4c92-a501-eabf2f3f1e63 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:10.438577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78b3c2-782f-4c92-a501-eabf2f3f1e63 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:10.438577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78b3c2-782f-4c92-a501-eabf2f3f1e63 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:10.442699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:10.448520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:10.448954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:10.449112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:10.467993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:10.469541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:10.477375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 00:13:10.500655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78b3c2-782f-4c92-a501-eabf2f3f1e63 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:10.513716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78b3c2-782f-4c92-a501-eabf2f3f1e63 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:10.517815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:10.518451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:10.518739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:10.519019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:10.527041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:10.528820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f42b6-3b4e-41b2-91d8-726cbcd152cb admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:10.537382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Jun 16 00:13:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:13:10.542738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:10.551622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccff24-4911-4e45-b6cf-3915220e1e04 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:10.556660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:10.557556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:10.557846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:10.558125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:10.559506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:10.561347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf5ece-ab20-45a4-ab09-59665ea0dbae admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:10.562902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Thu Jun 16 00:13:10 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:12.751283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07682-6d9f-40b1-ac0e-efc4c3c18eee None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:12.751747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07682-6d9f-40b1-ac0e-efc4c3c18eee None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:12.751852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a07682-6d9f-40b1-ac0e-efc4c3c18eee None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:12.753264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:12.756567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3c06-b59e-4346-a953-433f53819341 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:12.757035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3c06-b59e-4346-a953-433f53819341 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:12.757248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3c06-b59e-4346-a953-433f53819341 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:12.758403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094537b0-a642-42df-8646-e542c2751888 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:12.758698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094537b0-a642-42df-8646-e542c2751888 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:12.758984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094537b0-a642-42df-8646-e542c2751888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:12.759180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:12.763947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:12.764243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:12.764532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:12.783288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:12.784500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:12.800514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG passlib.handlers.bcrypt [None req-094537b0-a642-42df-8646-e542c2751888 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86548) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 00:13:12.800986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG passlib.handlers.bcrypt [None req-094537b0-a642-42df-8646-e542c2751888 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86548) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 00:13:12.846277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-094537b0-a642-42df-8646-e542c2751888 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:12.847846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-094537b0-a642-42df-8646-e542c2751888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:12.867241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 7/12] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:12.889762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:12.914042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:12.953304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 6/13] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:12.984629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094537b0-a642-42df-8646-e542c2751888 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:12.994796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab03f-9f9f-4991-a809-9385c49da662 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.028751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.029448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.029741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.030028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.083587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094537b0-a642-42df-8646-e542c2751888 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.085850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.096963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.098457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.121095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.121236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.122384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.122733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.123040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.135129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:13.146286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:13.147172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:13.147954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:13.153631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.198215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 8/15] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:13:13.203669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.253368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b88ce-de4f-46fd-b63f-45f61d5eef60 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.286202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.286868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.287155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.287432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.288195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:13.288503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.290695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.299937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185b96e-5196-486f-a664-8635ee3d04fa None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.304495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 8/16] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:13.310807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.326584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.343804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.344589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.344906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.345127 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.345215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.356496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.358017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:13:13.379754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:13.380592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:13.381548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:13:13.381548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:13:13.381548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:13:13.381548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:13.381548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:13.387217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.419512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd742664-87ad-4ff1-89df-1c8131d3c256 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.453655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.454531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.454818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.455096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.456662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.458160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.459776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:13.465715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.490851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7429fba-3084-4c55-9c11-12150561a9b8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.527190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.527789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.528076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.528356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.529110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:13.529424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.530822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.549093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:13.560062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.568202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1477ad-90e1-47fc-8076-4d875b44c161 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.606533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.607369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.607662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.607944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.609270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.610756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503182f8-7596-41d5-ab67-c566414dd5a4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.612429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:13.618677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:13.664444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:13.736551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9aac36-7221-4b47-a7f7-ed5194a0d944 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:13.770693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:13.771362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:13.771648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:13.771925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:13.772768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.domain.id=default, user.id=bfaf1438c4804d7499ba789f509be85e) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:13.773110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:13.775617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705b22e-8a04-483c-a856-e3a1f7868733 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:13.782749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Thu Jun 16 00:13:13 2022] GET /identity/v3/role_assignments?user.id=bfaf1438c4804d7499ba789f509be85e&scope.domain.id=default&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:16.110961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193f83-9726-4e3a-91a8-29b260b7e07f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.111975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193f83-9726-4e3a-91a8-29b260b7e07f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.112565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b193f83-9726-4e3a-91a8-29b260b7e07f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.114176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:16.119934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82e5c-714b-439b-a87d-8671bcaadcfe None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.120814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82e5c-714b-439b-a87d-8671bcaadcfe None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.121342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c82e5c-714b-439b-a87d-8671bcaadcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.144646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b0c82e5c-714b-439b-a87d-8671bcaadcfe None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:16.146284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b0c82e5c-714b-439b-a87d-8671bcaadcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:16.235684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:16.271905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193f83-9726-4e3a-91a8-29b260b7e07f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:16.337191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32f5c-1e8a-4ff8-bd52-502f18ab9f01 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.337612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32f5c-1e8a-4ff8-bd52-502f18ab9f01 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.337780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a32f5c-1e8a-4ff8-bd52-502f18ab9f01 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.339114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:16.357370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30554-393d-48b6-bc74-51f11a545ef5 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.357672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30554-393d-48b6-bc74-51f11a545ef5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.357951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a30554-393d-48b6-bc74-51f11a545ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.397928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-99a30554-393d-48b6-bc74-51f11a545ef5 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:16.400441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-99a30554-393d-48b6-bc74-51f11a545ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:16.401819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b193f83-9726-4e3a-91a8-29b260b7e07f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:16.460164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:16.486002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:16.486831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.487122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.487403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.537414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:16.538926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:16.543272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 14/25] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:16.552068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada523c2-d44a-4ded-a16b-55b4f522f7fc None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.552401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada523c2-d44a-4ded-a16b-55b4f522f7fc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.552731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada523c2-d44a-4ded-a16b-55b4f522f7fc None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.554172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 15/26] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:16.560372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.560667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.560977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:13:16.581142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:16.581992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:16.582767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:13:16.582767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:13:16.582767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:13:16.582767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:16.588527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 12/27] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:16.589476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:16.590826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:16.592029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:16.613149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:16.671320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 16/28] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:16.677722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:16.726346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66bec-b9df-4516-8ac2-7a6ce6d0e8e1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:16.772442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:16.773572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.775045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.775675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.787959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-855a6425-7c28-4fb4-a0b6-fbd92d466769 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:16.797440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:16.798456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:16.799319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:16.801093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:16.807438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 13/29] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:16.809092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33e79c-a2e9-4cf0-84d9-60b573587d41 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.809548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33e79c-a2e9-4cf0-84d9-60b573587d41 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.809975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33e79c-a2e9-4cf0-84d9-60b573587d41 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.816225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c84dd-70e8-4cd0-85b1-e5037e3272b4 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:16.816455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 14/30] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:16.817469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33e79c-a2e9-4cf0-84d9-60b573587d41 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:16.860385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:16.861228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.861614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.862017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.863349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:16.863764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:16.865722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:16.880776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 17/31] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:16.881919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:16.910280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:16.944343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33e79c-a2e9-4cf0-84d9-60b573587d41 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:16.988773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:16.989499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:16.989786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:16.990065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:16.990823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:16.991429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f975f-b592-462e-8ae4-cfcc70b1724b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:16.992211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.031784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.032952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.033469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.034054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.035097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:17.035745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.037541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc207f4-6f86-48e2-af18-108b073ffd5f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.052764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 18/32] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:17.054656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.055491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.056094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.082049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:17.083690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:17.087981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:17.156990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 15/33] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Jun 16 00:13:16 2022] POST /identity/v3/roles => generated 225 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:17.158872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:17.181277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 19/34] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:17.181277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:17.293367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07c100-972e-48dc-83b6-098a004ba150 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:17.330930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.331985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.332600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.333106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.334055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.335928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.341579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:17.361517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a16c0-06fe-46e9-a00f-cec31616f325 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:17.389174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 16/35] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:13:17 2022] POST /identity/v3/projects => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:17.390578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:17.495155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.496125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.496416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.496695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.498213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.499795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.501939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 20/36] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:17 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:17.511016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:17.537291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:17.642289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b41e0-b5af-44f9-b05f-c38d823f8191 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:17.691552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.692600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.693114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.693589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.694564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.697372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18e891-0524-414f-86f0-8ea611023635 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.697756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b0688-a1e7-4d0f-9913-66224d50663f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:17.722973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 17/37] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Jun 16 00:13:17 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:17.727086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c92cd-adae-489e-82c2-ada0d72abf87 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.728533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c92cd-adae-489e-82c2-ada0d72abf87 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.728872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c92cd-adae-489e-82c2-ada0d72abf87 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.730623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 18/38] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:17.746466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed77b-192c-4d3a-82d9-42401a54dd2a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.746823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed77b-192c-4d3a-82d9-42401a54dd2a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.747052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bed77b-192c-4d3a-82d9-42401a54dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.760934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.761772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.762070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.762357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.764653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.766200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.768084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 21/39] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:17.775391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:17.790131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-55bed77b-192c-4d3a-82d9-42401a54dd2a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:17.791965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-55bed77b-192c-4d3a-82d9-42401a54dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:17.865844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:17.895565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2db25-df23-405e-97b7-68a141b558b1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:17.935762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:17.936651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.936968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.937247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] PATH_INFO: `/v3/domains/default/users/bfaf1438c4804d7499ba789f509be85e/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.939308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bfaf1438c4804d7499ba789f509be85e, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:17.940789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:17.950684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:17.987336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 19/40] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:17.989286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0961-fb61-4e65-a5a2-c43ebc2735d7 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.989813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0961-fb61-4e65-a5a2-c43ebc2735d7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.990354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a0961-fb61-4e65-a5a2-c43ebc2735d7 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:17.991997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 20/41] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:17.998330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:17.998622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:17.998913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:18.017849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:18.019091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:18.089805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 21/42] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:18.090630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:18.164795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1361 bytes} [Thu Jun 16 00:13:17 2022] PUT /identity/v3/domains/default/users/bfaf1438c4804d7499ba789f509be85e/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:13:18.165970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:18.204207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4cb6-6240-4256-bca0-7ffb938cdb02 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:18.212691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:18.247843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:18.248549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:18.248837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:18.249153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:18.249992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:18.252008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:18.267737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Jun 16 00:13:18 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:18.282052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98c77a-1533-42e8-bf01-c02fb411ba34 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:18.322363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:18.323262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:18.323563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:18.323849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:18.325274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:18.327120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efb972-a535-49af-bd4e-e1b1fe6fd4b4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:18.331992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:18.337991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:18.455850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eccbe1-042c-4034-9630-8b3ed08b8e49 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:18.491244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:18.491917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:18.492205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:18.493369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:18.494166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:18.495632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27a589-ec16-455a-990a-da048ed45923 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:18.520788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:13:18 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:20.301938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead926d-4e0c-41b9-84b9-7910710fe2ef None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.302451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead926d-4e0c-41b9-84b9-7910710fe2ef None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.302532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead926d-4e0c-41b9-84b9-7910710fe2ef None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.303862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:20.309169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.309541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.309868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.326568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:20.327750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:20.341396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.395336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:20.419422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead926d-4e0c-41b9-84b9-7910710fe2ef None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.423303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.526080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead926d-4e0c-41b9-84b9-7910710fe2ef None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:20.559597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:20.560341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.560634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.560946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.575446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-1c96243d-ff63-487e-8345-05bde726a676 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:20.585038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:20.586109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:20.586916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:20.593984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.640169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bd57f-d7e3-4eaa-b3ed-213293205f24 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.640455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bd57f-d7e3-4eaa-b3ed-213293205f24 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.640748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bd57f-d7e3-4eaa-b3ed-213293205f24 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.642165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 26/50] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:20.647022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.647319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.647610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.667661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:20.669224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:20.698075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75c492-8cc6-42bd-9e77-bce3393e9dd7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:20.729226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.733527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:20.734221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.734512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.734793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.735588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:20.735901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:20.737335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:20.738116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 27/51] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:20.746767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 25/52] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:20.751946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.762048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.775421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.852890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e19f85-2af2-4be4-b67b-85efff16efe8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:20.864958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd78c-1cdf-47aa-855f-27ad902eb48d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:20.887142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:20.888057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.888350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.888643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.889893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:20.891351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:20.893025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 28/53] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:20 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:20.898269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.899166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:20.900225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:20.900514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:20.900795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:20.902173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:20.904395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:20.906050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 26/54] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:20.911546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:20.959584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.993612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:20.999623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1b362-b9ab-440b-ad8d-51feb344780d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:21.020505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a823a-137f-490a-acdb-4f335e054ecf None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:21.035838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:21.036666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:21.036979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:21.037259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:21.038452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:21.039920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:21.041798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 29/55] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:21.047213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:21.054187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:21.055385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:21.055770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:21.056050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:21.056767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:21.058236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9de4f-ab21-4e0c-8f07-36aff139e8ef None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:21.083935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 27/56] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Jun 16 00:13:20 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:21.155754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cae020-9913-41da-beae-e01988a11aae None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:21.186148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:21.192765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:21.193491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:21.193783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:21.194068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:21.194922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.domain.id=default, user.id=bfaf1438c4804d7499ba789f509be85e) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:21.195246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:21.197467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6c5d1-df6b-4c47-bd37-c8a3f99f4c95 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:21.204208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 30/57] 10.222.0.23 () {56 vars in 1322 bytes} [Thu Jun 16 00:13:21 2022] GET /identity/v3/role_assignments?user.id=bfaf1438c4804d7499ba789f509be85e&scope.domain.id=default&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:23.265911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20fa05-3068-4863-b37a-4cee2c027502 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:23.266421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20fa05-3068-4863-b37a-4cee2c027502 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:23.266527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20fa05-3068-4863-b37a-4cee2c027502 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:23.267839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 28/58] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:23.272977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:23.273383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:23.274091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:23.291252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:23.292423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:23.355582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 31/59] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:23.380268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20fa05-3068-4863-b37a-4cee2c027502 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:23.488740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20fa05-3068-4863-b37a-4cee2c027502 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:23.516805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:23.523279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:23.524169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:23.524717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:23.525030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:23.526221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:23.527667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3d5a9-09e5-48d7-a503-2c58205f30ca None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:23.529381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 29/60] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:23.535027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:23.634118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956c06-298b-4898-9fab-21356e00d748 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:23.656962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:23.668582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:23.669280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:23.669573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:23.669860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:23.670581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:23.672037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524912-e51b-4725-8c0d-63b99ef68ce4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:23.698263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 32/61] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:23 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:13:26.842133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe087-b9fd-459f-8ce5-545324ff3aa0 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:26.843137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe087-b9fd-459f-8ce5-545324ff3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:26.843621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe087-b9fd-459f-8ce5-545324ff3aa0 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:26.845231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 30/62] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:26.850382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0089-aaab-4ba9-90f8-78575bfd86b8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:26.851188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0089-aaab-4ba9-90f8-78575bfd86b8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:26.852214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0089-aaab-4ba9-90f8-78575bfd86b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:26.877651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-005a0089-aaab-4ba9-90f8-78575bfd86b8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:26.878888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-005a0089-aaab-4ba9-90f8-78575bfd86b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:26.967193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 33/63] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:27.003592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fe087-b9fd-459f-8ce5-545324ff3aa0 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.135043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d23ba9-d3e5-423a-883c-992c50b0413d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.135736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d23ba9-d3e5-423a-883c-992c50b0413d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.136219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d23ba9-d3e5-423a-883c-992c50b0413d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.137884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 34/64] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:27.141997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fe087-b9fd-459f-8ce5-545324ff3aa0 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.143732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb396-c92a-4bdc-8465-531b292541bd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.144238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb396-c92a-4bdc-8465-531b292541bd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.144717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bb396-c92a-4bdc-8465-531b292541bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.175685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.176497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.176788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.177092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.182167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-db7bb396-c92a-4bdc-8465-531b292541bd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:27.183733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-db7bb396-c92a-4bdc-8465-531b292541bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:27.188818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:27.197183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:27.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:27.198760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:27.201104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 31/65] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:27.209850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.282705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.315240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 35/66] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:27.329497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b32f0e-b7c1-47e1-afac-67c6ae1cec46 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.335532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999a7c3-6d03-411b-bfb2-8adb055ddc9f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.335835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999a7c3-6d03-411b-bfb2-8adb055ddc9f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.336252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999a7c3-6d03-411b-bfb2-8adb055ddc9f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.338330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 36/67] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:27.344929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.344929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.345168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.368943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:27.370620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:27.371652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.372551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.373018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.373480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.374443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:27.374963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:27.376590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:27.387048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 32/68] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:27.388282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.425241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.465055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 37/69] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:27.466306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.505399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebffe86-f539-4f38-9910-1231726062bb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.528916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.545701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.546456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.546749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.547036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.558150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-aaab1d93-bf0c-49b8-97f4-3b34bc07602c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:27.565861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:27.566712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:27.567441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 33/70] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:27.568526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd540-b275-4540-bdf2-3fc1695cae75 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.568617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd540-b275-4540-bdf2-3fc1695cae75 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.568861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd540-b275-4540-bdf2-3fc1695cae75 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.572990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 34/71] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:27.572990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd540-b275-4540-bdf2-3fc1695cae75 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.584388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586ce53-c34e-4aca-9e55-f80efe1149dd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.621309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.622560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.623065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.623595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.642387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:27.644308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:27.646414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 38/72] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:27.647609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.680052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.702557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd540-b275-4540-bdf2-3fc1695cae75 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.738515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.739321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.739608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.739890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.741096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:27.743095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e645da-e017-4046-a5de-bb6272999de5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:27.744700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 35/73] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:27.745935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27220a-4cec-456c-9e68-4994a34ccd50 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.746232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27220a-4cec-456c-9e68-4994a34ccd50 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.746864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27220a-4cec-456c-9e68-4994a34ccd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.761617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8d2a-7902-48ec-ba84-b6e2d4366bfc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.763848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dc27220a-4cec-456c-9e68-4994a34ccd50 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:27.765090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dc27220a-4cec-456c-9e68-4994a34ccd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:27.795999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.796610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.796921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.797211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.797970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:27.798286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:27.799695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:27.808233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 39/74] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:27.808702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.809060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.837103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 36/75] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:27.838488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fa61-a96d-412b-8490-41b546ee637b None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.838712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fa61-a96d-412b-8490-41b546ee637b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.838996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0fa61-a96d-412b-8490-41b546ee637b None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.840244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 37/76] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:27.840944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0fa61-a96d-412b-8490-41b546ee637b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:27.902777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:27.913174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4b45-de87-40f9-ae3c-4f13284c4457 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.945732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.946384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.946675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.946964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.952010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0fa61-a96d-412b-8490-41b546ee637b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:27.985971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:27.986642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:27.986928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:27.987205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:27.987923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:27.989400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.014073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:28.014495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 38/77] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:27 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:28.025239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.025333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003ba15-1977-4a73-8fde-86ca92cc5340 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.076179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.128650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.146037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 40/78] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:28.146037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.146037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.146037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.161421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb07d5-06a5-47f0-acec-12cb02616960 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.161973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:28.163204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:28.208640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.209663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.210385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.210679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.212925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:28.214016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.216038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:28.216850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.243261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 41/80] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:28.244194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.356021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0599bd-2898-4f1c-8740-b23925ec6ec0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.358610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.370043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6386a2-3d16-462d-98e9-c6f70e462f3c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.391209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.404101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.404961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.405258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.405637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.406855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.407785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.408070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.408347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.409622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:28.411073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.412853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 42/81] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:28.413782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.418202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:28.427706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:28.428565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:28.429396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:28.430229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 40/82] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:28.432011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.532441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c26199-5612-4f83-a067-a92cc32a3274 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.558565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edca847-e1fb-4121-a432-a4960c79325e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.568554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.569271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.569568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.569892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.599524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.600441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.600739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.601150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.614382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:28.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:28.624671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:28.626360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:28.629583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:28.630947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.635052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.645212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:28.646696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.657931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.755551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337dbcc-f64c-4241-8958-42285503211b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:28.787278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 43/84] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:28.788772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.796712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:28.797566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:28.797961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:28.798350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:28.799316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:28.799850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:28.801509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:28.816912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:28.816912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:28.963710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.986422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:28.987991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e4dfb-052f-416f-9edb-a13e96dbfec6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.025347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463453-8d4a-4c6c-9933-e53f5f2e1ae6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.034778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.036308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.036801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.037304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.038723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:29.040139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.042210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.052335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.057948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.094900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.096676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.097232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.097862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.100061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:29.101460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.103975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.130844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.133428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.234561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef2073-b272-4595-b2d5-5cf1bcfc6d3b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.300503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.301425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.301717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.302001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.303224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.305201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.307061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.307846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.308403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568deee-93b0-423e-b403-9d94503e7e2d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.344424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:29.345770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.346891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.347385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.347854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.349492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.351145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.358023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.359134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.380041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:29.505115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5e94-86ec-4428-9264-311500c72c67 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.540639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363efd26-503b-4ac4-8c44-87683f7fe98c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.541086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.541883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.542181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.542473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.544493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.550104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.551831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.552618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.621101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.622132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.622621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.623101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.624165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=bfaf1438c4804d7499ba789f509be85e) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:29.624681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.626370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.638919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/role_assignments?user.id=bfaf1438c4804d7499ba789f509be85e&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:29.640187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.648332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:29.668345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2635640-f3cf-4988-a2b8-c24372851884 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.711798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.712794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.713291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.713773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.744198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:29.767777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0584e9-75f8-4b02-bb77-8c8280ace051 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:29.796099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.798004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:29.813676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:29.814467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:29.814836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:29.815197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:29.904134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:29.924029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 47/92] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:29.930709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:29.942763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:29.949667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:30.039259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7a669-9394-437d-8388-0aaddff431c8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:30.085606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:30.087484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.087966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.088424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.089921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:30.091599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:30.093541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 48/93] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:30.096934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:30.099564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:30.145505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 46/94] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:30.152676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:30.197041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:30.273220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54180dc-fbcf-40c1-b5a5-09e635ebeb08 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:30.324414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:30.325414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.325886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.326381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.327416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:30.327918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:30.329587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:30.340908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 49/95] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:30 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:30.396812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c1b4-4538-4ec0-bf39-7317339519d3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:30.454726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:30.455630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.455922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.456207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] PATH_INFO: `/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.457585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] RBAC: Authorizing `identity:get_project(project_id=eeb798474dc24df2b2e9e5ae6efee58f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:30.459084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b89f1-e0bf-481e-b49e-5cc0481acebf None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:30.461389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 47/96] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:30 2022] GET /identity/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:30.467622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:30.619055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3347e6-87c5-408b-9b0b-51464683c443 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:30.639124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:30.669933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:30.671014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.671511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.672028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.673159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=eeb798474dc24df2b2e9e5ae6efee58f, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:30.673720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:30.675446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1ac8b-59cc-428f-9a45-24f127bd0126 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:30.683048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 50/97] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:30 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=eeb798474dc24df2b2e9e5ae6efee58f&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:30.792809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480a0e-f58d-42ce-8fa3-6c2568effc0f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.793144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480a0e-f58d-42ce-8fa3-6c2568effc0f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.793414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed480a0e-f58d-42ce-8fa3-6c2568effc0f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.795151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 48/98] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:30.800937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8624e6-d780-4673-81ae-a614fbf9449f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.801568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8624e6-d780-4673-81ae-a614fbf9449f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.802058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8624e6-d780-4673-81ae-a614fbf9449f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.824724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-bd8624e6-d780-4673-81ae-a614fbf9449f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:30.827241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bd8624e6-d780-4673-81ae-a614fbf9449f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:30.926309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 51/99] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:30.932826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fa2f-2dd5-485a-a308-95ef4c5495dc None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.933244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fa2f-2dd5-485a-a308-95ef4c5495dc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.933610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fa2f-2dd5-485a-a308-95ef4c5495dc None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.935193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 49/100] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:30.940796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:30.941417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:30.941878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:30.964860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:30.966387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:30.974316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fa2f-2dd5-485a-a308-95ef4c5495dc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.081335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:31.100598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 52/101] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:31.129284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.187184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fa2f-2dd5-485a-a308-95ef4c5495dc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:31.206850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:31.256736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:31.258426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.261083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.261573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.287146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:31.295258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:31.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:31.296823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:31.299179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 50/102] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:31.305151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.314717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1e18e-2255-44af-96f7-807f45630594 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:31.356248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:31.356248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.356248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.356248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.357453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:31.359165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:31.361625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:31.366889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.485545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68228b-f582-4acc-9136-9a68c8019660 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:31.504933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:31.527432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:31.532903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.532903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.532903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.532903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:31.532903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:31.556292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31504c-2810-4e8e-954e-4ed0d2d7f955 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:31.569002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:31.580911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 54/104] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:13:31 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:31.606224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:31.606929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.607262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.607547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.608395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] RBAC: Adding query filter params (name=reader) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:31.608713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:31.610324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:31.625330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 51/105] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:31.630916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.767415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f2de7-8f3e-422e-b57e-2e3eaa60c1ed None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:31.829692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:31.830766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.831059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.831340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] PATH_INFO: `/v3/users/253faa62969a43138f3215c611896c6c` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.851245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] RBAC: Authorizing `identity:get_user(user_id=253faa62969a43138f3215c611896c6c)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:31.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954ece3-6daf-4105-af07-77ddf3f700cb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:31.855281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 55/106] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:31 2022] GET /identity/v3/users/253faa62969a43138f3215c611896c6c => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:31.861235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:31.938118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2868ad-ee1d-4b7b-8b14-7bc5e153d8ba None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.938410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2868ad-ee1d-4b7b-8b14-7bc5e153d8ba None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.938700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2868ad-ee1d-4b7b-8b14-7bc5e153d8ba None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.940083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 56/107] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:31.947560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:31.947858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:31.948143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:31.981930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:31.983147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:32.004982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:32.029411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b609e-d560-4bcc-83a6-359702440abe None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.064182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.064842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.065160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.065439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.095790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 57/108] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:32.110597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:32.131940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.141568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:32.143065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:32.264078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 52/109] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:32.270110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.340158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155915-626d-4afe-9fb1-e6b5d83ec5fa None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.346145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:32.381015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be4170-4d1d-4ea2-bda6-8e51a4238dfd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.416635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.417718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.418027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.418311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.419144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.420047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.420415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.420706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.422045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:32.423515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:32.425287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 53/110] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:32.434712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.437070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:32.455399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:32.456336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:32.457144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 58/111] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:32.463483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.485315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:32.544564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166b821-7ed6-481b-b7ac-5c3036d9ab57 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.583058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.583734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.584130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.584421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.585322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=253faa62969a43138f3215c611896c6c) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:32.585746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:32.588410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:32.598059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 54/112] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/role_assignments?user.id=253faa62969a43138f3215c611896c6c&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=288574a645fb48129c87eb8509d301e7 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:32.632728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04222b6b-6b9e-4a30-a7c2-23591317dc50 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.680952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.681742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.682044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.682335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.683130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:32.683450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:32.684914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:32.694930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 59/113] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:32.700855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.726267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:32.844749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dab2a7-d9d0-4055-ae12-da068e71c2de None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:32.884672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:32.885530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:32.885826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:32.886110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:32.887342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:32.888796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:32.890586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 55/114] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:32.896062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:32.971837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:33.003964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417722f-2e87-47a2-9e33-2552f1c68ef5 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:33.039578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:33.040466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.040762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.041082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.042307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:33.043793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c9eb6-bcc5-40aa-939e-01226dd0d2ef None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:33.045720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 60/115] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:32 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:33.051375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:33.144911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:33.184993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb315b-2796-4dda-b894-a537d6257c67 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:33.211786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85077-f5a2-4b7c-b4ff-2560bb3ccd8c None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.212318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85077-f5a2-4b7c-b4ff-2560bb3ccd8c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.212535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc85077-f5a2-4b7c-b4ff-2560bb3ccd8c None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.214014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 61/116] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:33.219254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.219568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.219845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.248860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:33.250110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:33.259243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:33.260169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.260463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.260752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/5c4be08c17964e0f904d1c3c701380af/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.264803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, user_id=5c4be08c17964e0f904d1c3c701380af, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:33.266314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:33.301532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 56/117] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:33 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/5c4be08c17964e0f904d1c3c701380af/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:33.375374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 62/118] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:33.410974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:33.544819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9ed9-c04b-4172-8d2f-c353426eea15 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:33.576799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:33.583781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:33.585321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.585957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.586125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.597840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:33.606646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:33.607479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:33.608212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 57/119] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:33.614912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:33.745499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:33.778023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337f5e-7251-4dc0-a5e8-f1fa095a00ab None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:33.821688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:33.822379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:33.822677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:33.822959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:33.823725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:33.824039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:33.825457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ad70-b1dc-4d98-9f5d-1aeb01c3e3e7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:33.834483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 63/120] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:33.841062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.005348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3c2a-3dc2-42ea-9e64-ab5b2fb14870 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:34.046607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d634f1-5703-4250-9909-f2b3265dece8 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.046905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d634f1-5703-4250-9909-f2b3265dece8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.047356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d634f1-5703-4250-9909-f2b3265dece8 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.047767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:34.048694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.049020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.049314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.049654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 64/121] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:34.050723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:34.052249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:34.054287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 58/122] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:33 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:34.056542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374403-e759-4dc7-a14e-6560f5f38ae2 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.057106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374403-e759-4dc7-a14e-6560f5f38ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.057582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8374403-e759-4dc7-a14e-6560f5f38ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.064791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.097929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d8374403-e759-4dc7-a14e-6560f5f38ae2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:34.099479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d8374403-e759-4dc7-a14e-6560f5f38ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:34.169363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:34.172037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba7344-820b-4bd9-88c7-73981aa08c2c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:34.252272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:34.254128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.254739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.256375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.271359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 65/123] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:34.273216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed90-42e7-4999-bde8-ecac352b1622 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.273741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed90-42e7-4999-bde8-ecac352b1622 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.274171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3aed90-42e7-4999-bde8-ecac352b1622 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.277394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:34.279309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:34.280673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 66/124] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:34.282813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 59/125] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:34 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:34.285200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.285642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.286137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.293735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.334080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:34.335518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:34.460194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:34.495177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f12476-c4b7-4e5e-89f7-2d4a86bbc761 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:34.497889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 67/126] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:34.498881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.557101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:34.581598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:34.582958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.583438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.583960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/bfaf1438c4804d7499ba789f509be85e/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.593775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, user_id=bfaf1438c4804d7499ba789f509be85e, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:34.595600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:34.635100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 60/127] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:34 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/bfaf1438c4804d7499ba789f509be85e/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:34.636442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.660917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7c4b-5ac1-4a4c-8d89-3b60db857ff8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:34.769642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:34.770824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.771312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.771789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.781320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbedad-920c-483b-94b5-6da20d08f184 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:34.798450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:34.815413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:34.816351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:34.817117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 68/128] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:34.825947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:34.836428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:34.837611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:34.838206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:34.838624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:34.866223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:34.888262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:34.889267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:34.890014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:34.902641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 61/129] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:34.919921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.027855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:35.032809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:35.158303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21b201-ff44-42df-af7a-7854cb5a6660 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.172777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af229d4-21ad-4104-a1ee-680b9b4d037d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.230358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.231061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.231352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.231633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.232515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:35.232835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.234262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.252649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 69/130] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:35.254237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.254936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.255233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.255528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.256303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:35.256623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.257924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.258306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.282117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 62/131] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:35.291276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.451223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19953ff0-37b7-43f4-8e61-bf95d8a5541b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.465304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:35.485603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda00796-c965-4065-84e7-033333100934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.486777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00796-c965-4065-84e7-033333100934 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.487075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00796-c965-4065-84e7-033333100934 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.487358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00796-c965-4065-84e7-033333100934 None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.488575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda00796-c965-4065-84e7-033333100934 None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.490072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda00796-c965-4065-84e7-033333100934 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.496992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 70/132] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:35 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:35.498585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a878a-313a-4c44-818d-47300c13139b None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.498972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a878a-313a-4c44-818d-47300c13139b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.499304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a878a-313a-4c44-818d-47300c13139b None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.500653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 71/133] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:35.503047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a878a-313a-4c44-818d-47300c13139b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.544828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53efb4a-17c3-442f-8847-b40557d894c2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.568417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:35.637607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a878a-313a-4c44-818d-47300c13139b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.641727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616179da-8980-4274-a005-09732803f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.642621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616179da-8980-4274-a005-09732803f697 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.642910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616179da-8980-4274-a005-09732803f697 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.643191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616179da-8980-4274-a005-09732803f697 None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.651946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616179da-8980-4274-a005-09732803f697 None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.653465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616179da-8980-4274-a005-09732803f697 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.655168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 63/134] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:35 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:35.656515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.656812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.657123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.695810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:35.697058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:35.711074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.711821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.712109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.712387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] PATH_INFO: `/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.715799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] RBAC: Authorizing `identity:get_project(project_id=eeb798474dc24df2b2e9e5ae6efee58f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.717315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.719399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 72/135] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:35 2022] GET /identity/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:35.721275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.797898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:35.830986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 64/136] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:35.831841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.876507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46eeb44-5e8b-4bfe-b6c3-82b848dac04f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.912430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:35.913348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.913636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.914438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.915972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:35.917458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e3c70-421d-4fcf-be71-c6a53b310a3d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:35.919565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 73/137] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:35 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:35.925487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebe6e-b923-4a57-97bb-c2c6bbe929b7 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:35.925934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebe6e-b923-4a57-97bb-c2c6bbe929b7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:35.926220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ebe6e-b923-4a57-97bb-c2c6bbe929b7 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:35.927883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 74/138] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:35.928735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ebe6e-b923-4a57-97bb-c2c6bbe929b7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:35.976993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc6e97-2ac6-46a3-9292-9014c4ab933f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:35.984602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:36.016160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.017421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.017926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.018699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] PATH_INFO: `/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f/users/5c4be08c17964e0f904d1c3c701380af/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.021791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb798474dc24df2b2e9e5ae6efee58f, user_id=5c4be08c17964e0f904d1c3c701380af, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:36.025024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:36.045344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 65/139] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:35 2022] PUT /identity/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f/users/5c4be08c17964e0f904d1c3c701380af/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:36.046492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.064056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ebe6e-b923-4a57-97bb-c2c6bbe929b7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.097186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:36.128177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.129138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.129433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.129715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.133762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:36.135421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965b2d-1105-4d6c-b16d-b11f2da4405b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:36.141646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 75/140] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:36.145521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.147688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.152377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.194272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:36.195675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:36.215423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a162e-c22e-4555-9969-88f5da1b5b90 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.272689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.273183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.274036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.274327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/5c4be08c17964e0f904d1c3c701380af/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.278444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, user_id=5c4be08c17964e0f904d1c3c701380af, role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:36.279935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:36.297994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 66/141] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:36 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/5c4be08c17964e0f904d1c3c701380af/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:36.298963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.319520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 76/142] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:36.336981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:36.356385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.423997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c243-8129-43d8-bf1c-5b4457411c3d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.464036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.464739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.465056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.465340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.466185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:36.467761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd85d7-33b2-4940-bdec-dfea3f5cbd97 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:36.487452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674ddc7-94ac-4485-b001-7ba852259176 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.501475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 67/143] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:36 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:36.503007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305de8-8a24-4ff8-a972-49fc133ba793 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.503405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305de8-8a24-4ff8-a972-49fc133ba793 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.503697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305de8-8a24-4ff8-a972-49fc133ba793 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.505238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 68/144] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:36.511250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:36.512751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.513311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.513786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.533685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:36.534868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:36.550442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.551305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.551628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.551915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.563834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:36.576679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:36.578862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:36.580905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:36.586263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 77/145] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:36.592690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.615800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 69/146] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:36.644358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.686755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:36.766037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d1a9-f2fc-4058-92c7-15e482905fe8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.811390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.812221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.812512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.812794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.827706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:36.840024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:36.840943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:36.841713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:36.842290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 70/147] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:36.847756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.851958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41e13a-487c-4449-a647-a0c01a63eff1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:36.889324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:36.890011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:36.890306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:36.890590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:36.891362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:36.891738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:36.893941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:36.904742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 78/148] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:36.914327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:36.949249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.000917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55a1f-4c3d-457c-b700-47e6728cd7a2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.002791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.070600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.071211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.071505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.071788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.072565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:37.072912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.074346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.094656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 71/149] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:37.099990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:37.115626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d03d4-032d-4bc6-9086-d11ce093e19d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.221029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.221940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.222228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.222524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] PATH_INFO: `/v3/users/253faa62969a43138f3215c611896c6c` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.229721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] RBAC: Authorizing `identity:get_user(user_id=253faa62969a43138f3215c611896c6c)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.231197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.233106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 79/150] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:36 2022] GET /identity/v3/users/253faa62969a43138f3215c611896c6c => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:37.239278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:37.250058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3d52-9ebc-4446-8929-4c89d19e3c02 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.294155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.295103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.295394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.295676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.297087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.298728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.300395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 72/151] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:37 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:37.306001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:37.309008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.435033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abb6d5-d40c-4046-bf3c-6fe73fc4260a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.447104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07515b1f-ee0b-42ba-9889-344d46bcf55d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.447691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.484753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.486256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.486743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.487217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.490611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.493973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd22ab-f582-4142-9daf-b17bbd140f9c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.495742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.496351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 80/152] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:37 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:37.496664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.497092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.497528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.498430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447bf9a-c377-477b-9833-ab6b6c0dd55b None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.498875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447bf9a-c377-477b-9833-ab6b6c0dd55b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.499298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447bf9a-c377-477b-9833-ab6b6c0dd55b None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.501354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 81/153] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:37.503216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447bf9a-c377-477b-9833-ab6b6c0dd55b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:37.618934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.620578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.626826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.635092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447bf9a-c377-477b-9833-ab6b6c0dd55b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:37.675131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:37.676570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.677128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.677647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/253faa62969a43138f3215c611896c6c/roles/288574a645fb48129c87eb8509d301e7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.680984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, user_id=253faa62969a43138f3215c611896c6c, role_id=288574a645fb48129c87eb8509d301e7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:37.683047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156b54b-7cdc-437f-bb17-3812da5c4cc2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:37.709930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 82/154] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:37 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/users/253faa62969a43138f3215c611896c6c/roles/288574a645fb48129c87eb8509d301e7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:37.713381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:37.715176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:37.715938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:37.750738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:37.754870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:37.848068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 73/155] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:37.853884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:37.854526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:37.862317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 83/156] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:37.906953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:38.044814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1da8-a3e7-4ef0-835a-741fc71b40a4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.046344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:38.114298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.115192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.115487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.115770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.120446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:38.121960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:38.123735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 74/157] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:37 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:38.130675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:38.153785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889291c9-ab13-46c7-b536-6546ed818635 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.199294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.200124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.200417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.200723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.212208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:38.220347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:38.221363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:38.222125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 84/158] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:38.262863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:38.271058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f05343-f0cf-40bb-834b-00cdbe533c4f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.313409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:38.315715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.316401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.316693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.318031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.321909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:38.322242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:38.323744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:38.337097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 75/159] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:38 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:38.425129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76a43-0166-4b02-98b6-e224b7293b04 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.470168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.470861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.471152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.471434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.472208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:38.472527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:38.474029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:38.483380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 85/160] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:38.489348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:38.532158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:38.613342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f473e4-639d-446f-a7c4-9512cb72da36 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.669643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.670543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.670828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.671106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] PATH_INFO: `/v3/users/bfaf1438c4804d7499ba789f509be85e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.672377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] RBAC: Authorizing `identity:get_user(user_id=bfaf1438c4804d7499ba789f509be85e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:38.673860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:38.675653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 76/161] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:38 2022] GET /identity/v3/users/bfaf1438c4804d7499ba789f509be85e => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:38.682526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:38.819258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:38.837300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0dd54-3f41-4ba9-84eb-0495f601c3b1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:38.875428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:38.876019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:38.876309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:38.876595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:38.957555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:38.959110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:39.071429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:39.094541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 86/162] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:39.100375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:39.255783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810628c7-bb0e-4150-8a48-3d702f305c40 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:39.257313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:39.295838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:39.296712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:39.297021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:39.297302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:39.301599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:39.303072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7c029-32c3-4a82-aba8-fd220685695b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:39.305377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 77/163] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:39 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:39.313377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:39.466840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d905fab-bdac-4bf3-ab1c-b91e84e4a8d7 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:39.519011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:39.520055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:39.520542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:39.521053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:39.522124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=bfaf1438c4804d7499ba789f509be85e) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:39.522670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:39.525322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19f601-2f91-444d-9e22-27a78cb76fbb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:39.537615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 87/164] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:39 2022] GET /identity/v3/role_assignments?user.id=bfaf1438c4804d7499ba789f509be85e&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:39.682365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefc4f-62a0-451f-beb2-f878edd437c6 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:39.682900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefc4f-62a0-451f-beb2-f878edd437c6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:39.682900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefc4f-62a0-451f-beb2-f878edd437c6 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:39.684489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 78/165] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:39.699583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:39.700530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:39.701045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:39.737257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:39.738475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:39.794642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:39.817736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 88/166] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:39.842024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefc4f-62a0-451f-beb2-f878edd437c6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:39.982585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefc4f-62a0-451f-beb2-f878edd437c6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:40.020557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:40.021395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.021682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.021962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.037776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:40.045652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:40.046595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:40.047525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:40.051477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 79/167] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:40.053331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:40.210579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24788600-57b1-47c1-aa6f-76c94883b104 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:40.252543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:40.253279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.253565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.253846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.254607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:40.254915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:40.256301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:40.267672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 89/168] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:40.273006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:40.310649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:40.400134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0c06d-e482-46b9-a04a-397ad3d720fe None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:40.441766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:40.442701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.442988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.443267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.444467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:40.446646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd371036-09f1-4e2b-b60b-221b55658778 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:40.448509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 80/169] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:40 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:40.454060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:40.504512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:40.555888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4a96-245a-4876-8b0e-a28906c8b691 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.556206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4a96-245a-4876-8b0e-a28906c8b691 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.556466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba4a96-245a-4876-8b0e-a28906c8b691 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.557994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 81/170] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:40.563816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.564044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.564327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.572813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a3195-5827-4cd4-bbd0-a5ebcde871a6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:40.584181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:40.585405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:40.650920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:40.651715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:40.652008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:40.652290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:40.694615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 82/171] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:40.722563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:40.757329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:40.758977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:40.789672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:40.805932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:40.950328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 90/172] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:40.957394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:40.968052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3711a-d9a0-4013-834a-e29cfc9ddcd3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.050527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.051557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.052692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.053011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.081110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f6bdae6a-5a05-46b9-b749-d694cc04d9fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:41.090344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:41.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:41.092035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 83/173] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:41.098269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700b5b-71b0-4877-bf38-efe462e556e5 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.098559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700b5b-71b0-4877-bf38-efe462e556e5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.098842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00700b5b-71b0-4877-bf38-efe462e556e5 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.100283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 84/174] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:41.103569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00700b5b-71b0-4877-bf38-efe462e556e5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.194019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f3c21-24f7-4726-a212-6c72948caa00 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.238379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.239737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.240025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.240302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] PATH_INFO: `/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.242515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb798474dc24df2b2e9e5ae6efee58f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.244986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9848b82-aff7-469d-a57c-49ab3536fed8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.245234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.247285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 91/175] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:40 2022] GET /identity/v3/projects/eeb798474dc24df2b2e9e5ae6efee58f => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:41.248485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa109-b878-4f70-974b-b5d083db859c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.248771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa109-b878-4f70-974b-b5d083db859c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.249085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa109-b878-4f70-974b-b5d083db859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.253518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00700b5b-71b0-4877-bf38-efe462e556e5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.266816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-41faa109-b878-4f70-974b-b5d083db859c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:41.268307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-41faa109-b878-4f70-974b-b5d083db859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:41.271506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.293910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.294596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.294881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.295159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.296081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:41.296422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.298001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1d078-862c-4431-891b-94710f7ba068 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.307044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 85/176] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:41.312905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b256-c57d-4b1a-b98f-38f511f9c47c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.313216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b256-c57d-4b1a-b98f-38f511f9c47c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.313502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50b256-c57d-4b1a-b98f-38f511f9c47c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.314928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 86/177] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:41.315712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50b256-c57d-4b1a-b98f-38f511f9c47c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.347338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 92/178] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:41.348503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa109-b878-4f70-974b-b5d083db859c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.455095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50b256-c57d-4b1a-b98f-38f511f9c47c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.458823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa109-b878-4f70-974b-b5d083db859c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.493878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.494476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.494738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.495017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.495856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=eeb798474dc24df2b2e9e5ae6efee58f, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:41.496170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.497631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf98c3f-e02d-4e5b-98dd-d725f93ef5e9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.498971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.500142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.500764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.501325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] PATH_INFO: `/v3/users/5c4be08c17964e0f904d1c3c701380af` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.502734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] RBAC: Authorizing `identity:get_user(user_id=5c4be08c17964e0f904d1c3c701380af)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.503838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 87/179] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=eeb798474dc24df2b2e9e5ae6efee58f&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:41.504763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.505163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.505452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.505728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.507656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 93/180] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/users/5c4be08c17964e0f904d1c3c701380af => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:41.508794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.521111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.533020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:41.534523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:41.546118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.617693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 88/181] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:41.618803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.640724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f526c2-5830-471f-8365-3be4bfbffd1a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.680763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.681874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.682307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.682860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.695885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:41.704916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:41.706026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:41.706999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:41.709216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 94/182] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:41.710170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.770618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e305-7c7d-476f-91dd-c0550675e82e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.800928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.821395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.822115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.822404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.822683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.840723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:41.841397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c1207-a81c-4077-91c9-42e7f8a2b746 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:41.880779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:41.881901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:41.882010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:41.883378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:41.884715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.886264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.888054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 95/183] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:41.888909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:41.909159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:41.910637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efb1c2-13cc-47a3-8fb1-70bdf7a13181 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:41.990343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7edba-c6dd-43cc-bd5c-9c96aca90d50 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.010669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 89/184] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:42.011987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1f10-f47c-46b0-adac-eececb77b291 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.012276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1f10-f47c-46b0-adac-eececb77b291 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.012558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1f10-f47c-46b0-adac-eececb77b291 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 90/185] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78921b-2df5-4755-9537-fd3a67d0ace7 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78921b-2df5-4755-9537-fd3a67d0ace7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78921b-2df5-4755-9537-fd3a67d0ace7 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 91/186] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:42.017952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78921b-2df5-4755-9537-fd3a67d0ace7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.030569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.031184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.031477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.031753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.032510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:42.032822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.034223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.036541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.043383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 96/187] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:42.046311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.067145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.122131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78921b-2df5-4755-9537-fd3a67d0ace7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.144020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59f078-8cf2-46a6-b60f-83a2839c3901 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.155228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.155899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.156182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.156454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.157193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.158634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64d7ec-26af-40e1-9ee9-2b609586fb50 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.177902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.178095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 92/188] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:42 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:42.178770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.179051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.179330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.181200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.182665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a296b1-4a0c-46b9-97b3-9b5763d2de8d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.184445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 97/189] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:42.185701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.185991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.186268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.186434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.186723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.187007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.204420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:42.205685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:42.207391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:42.208608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:42.274308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.283159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 98/190] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:42.283928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.294243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 93/191] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:42.294243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.330388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.419589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e195ae-4eef-4c0c-8de2-4fa4d9e3b729 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.419777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753fd98-ff18-44a9-8a73-03ce44a9e09b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.463478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.464162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.464449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.464725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.465089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.465625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=5c4be08c17964e0f904d1c3c701380af) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:42.465957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.466054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.466246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.466519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.467410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.474196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 94/192] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/role_assignments?user.id=5c4be08c17964e0f904d1c3c701380af&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:42.475119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.486257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.487775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.489816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 99/193] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:42.490447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.557755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.612446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.614645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f16d1-2f21-4736-bfa5-618938dbb6c8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.617213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75970c-a19f-4a88-9193-2510a4fc6d9b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.660830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.662199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.662493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.662776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.666850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.667696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.667983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.668288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.679590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:42.684418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:42.694573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:42.696024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:42.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:42.701077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:42.702359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:42.703859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 100/194] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:42.706012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 95/195] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:42.706248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.706488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.837867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b694e-3d6f-414e-b8f4-806f56c64256 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.842362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eac2a-ceab-4048-aa98-fa4c2e2b96ec None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:42.860044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.886217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.886941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.887238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.887526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.892894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:42.893618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:42.893912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:42.894194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:42.894962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:42.895275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.896677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:42.909167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 96/196] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:42.909850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:42.934064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:42.971914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:42.973716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.032762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887e84-0d25-427e-b219-e3ccb42731a2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.068141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.068832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.069155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.069436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.070289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:43.070678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.072164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.080585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 97/197] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.081439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.083982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 101/198] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:43.089863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.106303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.163595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.187904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255fdbb-d2ae-40c8-a075-65efff34386e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.198940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12a13a-2b03-4756-8ebe-9fa99157258c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.223503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.224657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.225102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.225516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] PATH_INFO: `/v3/users/253faa62969a43138f3215c611896c6c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.227400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] RBAC: Authorizing `identity:get_user(user_id=253faa62969a43138f3215c611896c6c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.229003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.230787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 98/199] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/users/253faa62969a43138f3215c611896c6c => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.231681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.238319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.239211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.239499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.239782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.241036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.243039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.245063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 102/200] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.246205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.334692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.338006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f022bb-e2b8-48a5-a3fd-419c601e51b4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.356298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605ad0c-d39d-4a05-a78b-aba943dec5f3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.373361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.374228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.374515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.374790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.375976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.377445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.379351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 99/201] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.379888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.394752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.395667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.396133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.396589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.396744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.470629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.472252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.487191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa16092-7b82-4ae1-8bd2-a68c2c715f19 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.522397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.523888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.524382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.525091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.563192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.585426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 103/202] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:43.587136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.598139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.599828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.615891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.701499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3c50a-e182-437e-9265-29eb8cf4173b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.708457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 100/203] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:43.709497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.741153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.741826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.742653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.742934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.743786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:43.744101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.745581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.751420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 104/204] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/role_assignments?user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.756968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:43.850115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.904041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ae6e9-354e-4664-891e-ccfe97aaa069 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.941767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:43.957574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922c39b-c1ae-4675-abe1-a5e40a784a99 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:43.969408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:43.970601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:43.970998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:43.971282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:43.972496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:43.977754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:43.977754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 101/205] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:43.983374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:44.001190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:44.002401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.002685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.002960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.005100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:44.011197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:44.022504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 105/206] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:44.026648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:44.180800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b0694-8664-4485-9da2-128164aa7042 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:44.205279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2688e-8598-41d5-a122-f1a37ad729f1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:44.241267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:44.252365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:44.253079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.253366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.253640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.254489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:44.254802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:44.256242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84587a-1032-48e2-aa01-1b759e9751cd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:44.262599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 106/207] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:44 2022] GET /identity/v3/role_assignments?user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:44.274997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:44.275722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.276015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.276294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.277225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7, user.id=253faa62969a43138f3215c611896c6c) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:44.277546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:44.279088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f08a38-fe63-47d9-8b38-af5acd838560 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:44.299368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 102/208] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:43 2022] GET /identity/v3/role_assignments?user.id=253faa62969a43138f3215c611896c6c&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=288574a645fb48129c87eb8509d301e7 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:44.654564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be343-8082-419f-9ea1-fc2ec1bdc964 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.654978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be343-8082-419f-9ea1-fc2ec1bdc964 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.655156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be343-8082-419f-9ea1-fc2ec1bdc964 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.656707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 107/209] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:44.662656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac170-7695-4407-b3d5-f005f4d0de9d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.662942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac170-7695-4407-b3d5-f005f4d0de9d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.663233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac170-7695-4407-b3d5-f005f4d0de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.681195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8a9ac170-7695-4407-b3d5-f005f4d0de9d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:44.682414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a9ac170-7695-4407-b3d5-f005f4d0de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:44.730103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:44.756515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 103/210] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:44.784429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0be343-8082-419f-9ea1-fc2ec1bdc964 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:44.908903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0be343-8082-419f-9ea1-fc2ec1bdc964 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:44.949804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:44.950608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.950888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.951158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.961881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:44.967756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4e94-0185-4c75-abe5-9bc62f8da5ec None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.968227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4e94-0185-4c75-abe5-9bc62f8da5ec None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.968298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf4e94-0185-4c75-abe5-9bc62f8da5ec None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.969855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 104/211] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:44.972098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:44.972969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:44.973775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:44.974801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:44.974894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:44.975170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:44.977233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 108/212] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:44.982926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:44.996625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:44.998258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:45.035792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:45.080751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 105/213] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:45.110557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:45.120940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01999f7b-b1b0-4793-8b3e-1d51d2adc1fa None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.164505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.165544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.166083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.166555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.168085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:45.168899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.171189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.184029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 109/214] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:45.189360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:45.225406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc59a4-af3f-489a-8d7a-b9316734d8fc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.235781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:45.269750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.271306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.271798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.272271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.274607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.277128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.283884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 106/215] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:45.284934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:45.302370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:45.306744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3857f-10a6-4369-8ddf-1f292ba6bbc8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.349091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.351156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.352308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.356740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] PATH_INFO: `/v3/users/0ed95d70ce2a4f60b84d735219bd40e4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.373431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed95d70ce2a4f60b84d735219bd40e4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.375167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.377229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 110/216] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:45 2022] GET /identity/v3/users/0ed95d70ce2a4f60b84d735219bd40e4 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:45.383770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:45.412818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43057f-ece4-4c6e-a4fc-1e5451f64fbd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.458089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.459485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.460200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.460643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.461570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.464153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.496930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 107/217] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:45 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:45.530228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567dcfd-88aa-417e-b911-8bb6c69ea602 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.572993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.574053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.574540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.575038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.612743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:45.661824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.663313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.790329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 111/218] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:13:45.796298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:45.835132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:45.920871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e18f0-42d0-4e22-a4b5-2e3139d29128 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:45.957463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:45.958351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:45.958639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:45.958922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:45.960150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:45.962236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35043cf-77a6-44d4-b861-1194d430f17a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:45.964352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 108/219] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:45 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:45.970221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:46.039305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:46.089847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd3600-9dcb-465e-bbe3-c65076b537c4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:46.146857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:46.147795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.148771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.149231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.150550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=0ed95d70ce2a4f60b84d735219bd40e4) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:46.151023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:46.152686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c27f2-180e-4aa4-be7c-98e58093f82e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:46.166202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 112/220] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:45 2022] GET /identity/v3/role_assignments?user.id=0ed95d70ce2a4f60b84d735219bd40e4&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:46.200742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331bc1-1c3f-4dea-99b7-0a521bdb6763 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.201373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331bc1-1c3f-4dea-99b7-0a521bdb6763 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.201587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8331bc1-1c3f-4dea-99b7-0a521bdb6763 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.203290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 109/221] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:46.209334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.209839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.210304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.239138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:46.240572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:46.345909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 113/222] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:46.382136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331bc1-1c3f-4dea-99b7-0a521bdb6763 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:46.526820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:46.531717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8331bc1-1c3f-4dea-99b7-0a521bdb6763 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:46.569206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:46.570603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.570895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.571171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.588854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c84ff1e3-4921-4d50-8f74-5f11aef937a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:46.596719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:46.597617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:46.598983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 110/223] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:46.603917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:46.669591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:46.705612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5b25d-0208-4b3d-b599-2b857f9e9a5b None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.705916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5b25d-0208-4b3d-b599-2b857f9e9a5b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.706174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5b25d-0208-4b3d-b599-2b857f9e9a5b None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.707691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 111/224] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:46.719872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.720155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.720438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.728336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044fd23-881f-4b46-a407-9bfc6441b26f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:46.765181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:46.765866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:46.766156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:46.766439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:46.767202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:46.767621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:46.769061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:46.771701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:46.772930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:46.780008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 114/225] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:46.785513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:46.911331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 112/226] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:46.945876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:46.948087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d0808-f70f-4dda-9fde-7845e4ac5230 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:46.981227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:47.018777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.019698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.019991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.020273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.021619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.023290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.025613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 115/227] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:46 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:47.033574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:47.110976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:47.135290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd114-54a7-43d1-8655-65c0a4236022 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.203063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.203821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.204094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.204377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.220226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18946f-3294-4488-8715-f3f06113484a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.226432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:47.239196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:47.240061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:47.240843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 113/228] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:47.247807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:47.292082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.292986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.293281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.293571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.296262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.297759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.308429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 116/229] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:47 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:47.323070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:47.412581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:47.445856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89affab-0b59-422f-a452-985ad0f09cb4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.484945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.485551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.485845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.486128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.486892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:47.487208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.488602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.498358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 114/230] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:47.506108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:47.514204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519501e2-3e3f-46d7-86c2-1b8dcf9de125 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.525551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:47.594595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.595510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.595795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.596179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/b7c7d9f9a2c04aec9bc63b3ea2b18841/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.598276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.599726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.622615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58141a10-cb54-43a9-a935-4808e8390cbd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.629803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 117/231] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:47 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/b7c7d9f9a2c04aec9bc63b3ea2b18841/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:47.665739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.666666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.666961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.667246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.672446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.672446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.673741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 115/232] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:47 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:47.679498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:47.859655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c9b7-1181-4cc7-bd01-065ff2ebff3e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:47.870967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:47.897683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:47.899772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:47.900316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:47.900597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:47.901837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:47.903315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee644-b03e-4f93-8407-3bf955ab581f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:47.909222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 118/233] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:47 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:47.913490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.016154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:48.058098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed1cc1-d76e-47ee-a381-43726cfe1dff None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:48.107011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:48.107918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.108207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.108489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/b7c7d9f9a2c04aec9bc63b3ea2b18841/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.110852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841, role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:48.112243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a713d5-ff59-45d4-81e2-5b3342f3419c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:48.134382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 116/234] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:47 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/b7c7d9f9a2c04aec9bc63b3ea2b18841/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:48.224759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0ee34-cf12-4059-87f7-37878ecc2cd8 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.225318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0ee34-cf12-4059-87f7-37878ecc2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.225393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0ee34-cf12-4059-87f7-37878ecc2cd8 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.227110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 119/235] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:48.242475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b406c6-0ac7-4c50-a021-be580b56851d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.243505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b406c6-0ac7-4c50-a021-be580b56851d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.243805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b406c6-0ac7-4c50-a021-be580b56851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.270679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-44b406c6-0ac7-4c50-a021-be580b56851d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:48.271875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-44b406c6-0ac7-4c50-a021-be580b56851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:48.377695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de79bb-0513-49f1-8626-1a5298122686 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.378334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de79bb-0513-49f1-8626-1a5298122686 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.378815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de79bb-0513-49f1-8626-1a5298122686 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.378980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 117/236] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:48.380786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:48.387130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.387424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.387710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.408521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de79bb-0513-49f1-8626-1a5298122686 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.410536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:48.411743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:48.498273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:48.505714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 118/238] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:48.535036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.570794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de79bb-0513-49f1-8626-1a5298122686 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:48.652955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3b77b-c5ad-4b54-8c08-64c2c6aa972d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:48.661573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:48.662396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.662689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.662973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.687885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:48.696800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:48.697628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.697917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.698196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:48.701577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:48.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:48.703211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 121/239] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:48.703866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:48.707440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:48.707440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 119/240] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:48.716937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.717138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.798445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:48.855786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:48.917857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0677d3-0ec1-4202-a583-35c48fea18ff None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:48.924833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999ab91-7a49-42bd-b1a9-d8ce8f4d205a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:48.962556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41145438-f378-4efd-9189-a5590c36192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:48.963814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145438-f378-4efd-9189-a5590c36192e None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.964254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:48.964574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145438-f378-4efd-9189-a5590c36192e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.965060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145438-f378-4efd-9189-a5590c36192e None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.965161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.965462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.965818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.966393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145438-f378-4efd-9189-a5590c36192e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:48.966728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:48.967109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:48.968408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145438-f378-4efd-9189-a5590c36192e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:48.968560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e311-1ad5-4416-a6b7-ab3991dc88f1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:48.978111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 122/241] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:48.979624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b507f-8262-4a3d-9606-5192a504d464 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:48.979992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b507f-8262-4a3d-9606-5192a504d464 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:48.980348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b507f-8262-4a3d-9606-5192a504d464 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:48.982132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 123/242] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:48.983403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b507f-8262-4a3d-9606-5192a504d464 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:48.997733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 120/243] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:48 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:48.999353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.000054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.000420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.020327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:49.021791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:49.103235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 121/244] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:49.109425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.117876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b507f-8262-4a3d-9606-5192a504d464 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.139882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.156244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.157199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.157486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.157763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] PATH_INFO: `/v3/users/211f5fb7e54c4a1687f434d798dfd4f7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.173544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] RBAC: Authorizing `identity:get_user(user_id=211f5fb7e54c4a1687f434d798dfd4f7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.175364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.177541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 124/245] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:48 2022] GET /identity/v3/users/211f5fb7e54c4a1687f434d798dfd4f7 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:49.183273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.234384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.294768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5e60a-a96d-4735-8c8b-5d417218dd72 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.296311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dff661-fd06-4116-bc54-ea1c9b62e888 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.332078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.332990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.333466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.333839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.336212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.337227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.337663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.338173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.350548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:49.359319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:49.360267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:49.361105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:49.363272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 122/246] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:49.368651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.370454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.409057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.410673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.489139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53369-1b9f-4424-bc2a-c63fc1dab4a1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.505352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.530476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 125/247] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:13:49.536911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.537162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.537416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.537695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.538460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:49.538839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.538967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.540327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.550466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 123/248] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:49.556127 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.605883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.651324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ff2e-4bb3-4543-9e43-ac57c850d0af None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.690370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477cc64-f2d1-40d1-97e0-cefa46e30ee7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.691020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.692249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.692857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.693348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.694814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.696912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.699343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 126/249] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:49.711350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.731693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.732595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.732904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.733246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] PATH_INFO: `/v3/users/0ed95d70ce2a4f60b84d735219bd40e4` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.734607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] RBAC: Authorizing `identity:get_user(user_id=0ed95d70ce2a4f60b84d735219bd40e4)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.736164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.738309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 124/250] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/users/0ed95d70ce2a4f60b84d735219bd40e4 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:49.744461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:49.811368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.846560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a118-f73a-4228-a2c3-4ebfe3af9c98 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.877004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b56a56-64d4-4784-8494-0e2acf2ba8ff None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:49.881389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:49.893116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.894458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.895169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.895641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.896838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=211f5fb7e54c4a1687f434d798dfd4f7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:49.897658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.899587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.907650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 127/251] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/role_assignments?user.id=211f5fb7e54c4a1687f434d798dfd4f7&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=288574a645fb48129c87eb8509d301e7 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:49.923510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:49.924435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:49.924729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:49.925103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:49.926358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:49.927865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecf74-8af9-43a6-9d8b-cfb3cd6b6ff3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:49.930023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 125/252] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:49 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:49.936207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:50.051826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2868807-c3bc-4587-900a-de153d8eeab2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:50.100365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:50.101914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:50.102206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:50.102488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/0ed95d70ce2a4f60b84d735219bd40e4/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:50.105951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, user_id=0ed95d70ce2a4f60b84d735219bd40e4, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:50.107426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e6a7e-77e8-4501-9807-8aaa64d6309f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:50.135913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 128/253] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:49 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/0ed95d70ce2a4f60b84d735219bd40e4/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:13:50.627871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ac80-776e-477d-ab3b-410c8cac6f22 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:50.628378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ac80-776e-477d-ab3b-410c8cac6f22 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:50.628451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5ac80-776e-477d-ab3b-410c8cac6f22 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:50.630116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 126/254] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:50.635369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:50.635862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:50.635939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:50.661913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:50.663146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:50.731777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:50.767131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 129/255] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:50.818157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ac80-776e-477d-ab3b-410c8cac6f22 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:50.958391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5ac80-776e-477d-ab3b-410c8cac6f22 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:50.969717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:51.002532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:51.003345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.003634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.003914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.021760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:51.030686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:51.031610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:51.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 127/256] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:51.037843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:51.167366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1084b9a-43ef-49fd-b0c6-0b8c2913f1ce None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:51.214360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:51.214963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.215256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.215542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.216307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:51.216618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:51.218046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:51.228314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 130/257] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:51.233665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:51.359351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000df797-38ac-4b2c-8974-31fe5cd5416e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:51.400826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:51.401741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.402047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.402311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.405647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:51.407109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c3a68-946a-4f38-a27d-54fea9e2e85d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:51.408764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 128/258] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:51.414938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:51.455589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310492f-b351-4d2e-bf55-43c60aad3dc4 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.455929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310492f-b351-4d2e-bf55-43c60aad3dc4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.456204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310492f-b351-4d2e-bf55-43c60aad3dc4 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.457745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 129/259] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:51.474591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.474839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.475087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.484032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:51.514189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:51.515380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:51.561613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545ba43-662b-41c2-b690-318e68b2ec29 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:51.607600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:51.628233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:51.628988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.629286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.629570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.674162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 130/260] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:51.710354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:51.712200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:51.713777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:51.787619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:51.853720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 131/261] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:51.859341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:51.865056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0060c3f-def8-480e-8be7-659bc3f50523 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:51.930983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:51.932291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:51.932769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:51.933281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:51.944494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:51.954598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:51.955491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:13:51.956271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:51.959221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 131/262] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:51.967689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.002955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:52.098243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cdb17-25d5-4268-bf41-3ace5ea8faa3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:52.126803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6a68-f6f3-4dcd-a905-ec577240970e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:52.187551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:52.188239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.188525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.188792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.189586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:52.189898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:52.191277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:52.209672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:52.210708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.211183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.211614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.214034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 132/263] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:52.214790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:52.216255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:52.218273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 132/264] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:51 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:52.223924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.225124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.254731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:52.450634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977edfc-a84f-493b-a1ee-4462af450a3a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:52.463792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:52.488918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453774-4c50-482d-91e5-c7af1e82955b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:52.491407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:52.493327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.493816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.494314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] PATH_INFO: `/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.495715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] RBAC: Authorizing `identity:get_user(user_id=b7c7d9f9a2c04aec9bc63b3ea2b18841)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:52.497572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb275ce-7f6a-4834-8ede-55427974031c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:52.499418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 133/265] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:52 2022] GET /identity/v3/users/b7c7d9f9a2c04aec9bc63b3ea2b18841 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:52.500972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fe7f-6493-4093-8263-ed8989fd9f0e None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.501470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fe7f-6493-4093-8263-ed8989fd9f0e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.501933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2fe7f-6493-4093-8263-ed8989fd9f0e None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.503421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 134/266] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:52.508539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2fe7f-6493-4093-8263-ed8989fd9f0e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.593838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:52.594302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.594595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.594878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.595797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:52.596058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:52.601309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:52.602060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc3f1-c049-4386-bd36-41d6621e9422 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:52.619570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2fe7f-6493-4093-8263-ed8989fd9f0e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:52.626907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 133/267] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:52 2022] GET /identity/v3/role_assignments?user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:52.628210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.628515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.628791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.660057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:52.661152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:52.661661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:52.662267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:52.677157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:52.678175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:52.751206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:52.753512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:52.850718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:52.869420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 135/268] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:52.875185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.894839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 134/269] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:52.932314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:52.990746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb525-8d41-4538-bbb1-9eea2b2c0e43 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.035954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.037328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.038318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.039620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.043310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.045183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58400e04-e0ae-4208-a5b4-4e160491fb74 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.048332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 136/270] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:52 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:53.049780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a28cd-7b50-4387-b103-c5fd82060d36 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.050210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a28cd-7b50-4387-b103-c5fd82060d36 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.050646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a28cd-7b50-4387-b103-c5fd82060d36 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.052140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 137/271] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:53.061953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a28cd-7b50-4387-b103-c5fd82060d36 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.070717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.089989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa424-78a8-4adf-a21e-307cec87fd9b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.118788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.129167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.130316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.130600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.130872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.144222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-65c9eac8-fb9e-4aff-909a-7e4bf70d711f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:53.154921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:53.156011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:53.156956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:53.159263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 135/272] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:53.160723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.161118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.161321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.184551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a28cd-7b50-4387-b103-c5fd82060d36 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.188700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:53.189994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:53.224689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.225634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.226085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.226966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.227943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:53.229123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.231335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.237809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 138/273] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/role_assignments?user.id=b7c7d9f9a2c04aec9bc63b3ea2b18841&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:53.238755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.330798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 136/274] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:53.360944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.418149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f91c4c-a8c8-4280-ab6d-d6931dac1e76 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.450555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.456924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.457718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.458321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.458666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.459521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:53.459844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.462286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b116-084a-4595-baff-05704082cfbe None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.468048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.474427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 139/275] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:53.475811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4871c-b50e-4902-b834-c0dd0888a05c None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.476315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4871c-b50e-4902-b834-c0dd0888a05c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.476824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4871c-b50e-4902-b834-c0dd0888a05c None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.478350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 140/276] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:53.482544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831573a5-df1e-4209-a76d-5209d565b080 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.483102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831573a5-df1e-4209-a76d-5209d565b080 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.483396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831573a5-df1e-4209-a76d-5209d565b080 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.486129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 141/277] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:53.486239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831573a5-df1e-4209-a76d-5209d565b080 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.498840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9332d41-352e-42b1-986f-9d86ca1b56b6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.542540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.543345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.543630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.543908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.545097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.546557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c95ac-4ae6-4cbd-a169-f70e1220f25e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.548362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 137/278] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:53.549360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef48e7-1c41-4d0a-aeed-4f8ebff5dd3c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.549648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef48e7-1c41-4d0a-aeed-4f8ebff5dd3c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.549928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef48e7-1c41-4d0a-aeed-4f8ebff5dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.566418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b9ef48e7-1c41-4d0a-aeed-4f8ebff5dd3c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:53.567596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b9ef48e7-1c41-4d0a-aeed-4f8ebff5dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:53.604847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831573a5-df1e-4209-a76d-5209d565b080 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.635012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 138/279] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:53.647207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.647207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.647207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.647426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.647426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.647426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.647426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] PATH_INFO: `/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.653977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe0d4ae4a64b3a904d657ad136b76e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.655434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.662142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 142/280] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:53.662142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.662339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:53.662339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:53.692447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.729553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 139/281] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:53.730399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.732633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.772056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf57ffd-c36d-4500-b80b-5eaf6fa46066 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.807485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.808245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.808597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.808996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.809774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:53.811211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:53.830704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 143/282] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:13:53 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:53.831369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a395dc-51dd-4141-a919-790055f7f48c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.831553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.876064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:53.876895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:53.877193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:53.877468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:53.889566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:53.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:53.899009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:53.899800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 140/283] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:53.900736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:53.944705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:53.963496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a7f5e-10bf-4281-9034-157bd996d10a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:53.970294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.002390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.005654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.005948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.006241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.007099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, user.id=8dbe0d4ae4a64b3a904d657ad136b76e, scope.system=all) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:54.007508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.009133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.017796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 144/284] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/role_assignments?user.id=8dbe0d4ae4a64b3a904d657ad136b76e&scope.system=all&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:54.017981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.029655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235d69-0ac4-4d84-8c01-19e8c8269e13 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.077250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.078495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.078778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.079103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.092783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:54.101713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:54.102669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:54.103477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 141/285] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:54.111566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.139821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ae06a-765f-478e-afaa-e5a1ae3def27 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.186867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.187747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.188115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.188401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.189759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:54.189759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.190998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.203894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 145/286] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:54.213752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.231461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.249839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75bc669-0a00-4760-9d02-e51315eedb41 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.281722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.293949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24293906-97de-40f5-a973-ed29a957919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.295311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293906-97de-40f5-a973-ed29a957919c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.295612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293906-97de-40f5-a973-ed29a957919c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.295899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24293906-97de-40f5-a973-ed29a957919c None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.296677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24293906-97de-40f5-a973-ed29a957919c None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:54.297033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24293906-97de-40f5-a973-ed29a957919c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.298713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24293906-97de-40f5-a973-ed29a957919c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.310680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 142/287] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:54.317276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24293906-97de-40f5-a973-ed29a957919c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.342730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0dc0c-d38f-425c-abed-20929a5a4a8d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.380166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.381270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.381692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.382030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] PATH_INFO: `/v3/users/211f5fb7e54c4a1687f434d798dfd4f7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.383909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] RBAC: Authorizing `identity:get_user(user_id=211f5fb7e54c4a1687f434d798dfd4f7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.385612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.387399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 146/288] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/users/211f5fb7e54c4a1687f434d798dfd4f7 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:54.392735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.428190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24293906-97de-40f5-a973-ed29a957919c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.465760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.466591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.466873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.467151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] PATH_INFO: `/v3/users/0ed95d70ce2a4f60b84d735219bd40e4` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.468411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed95d70ce2a4f60b84d735219bd40e4)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.469945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.471802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 143/289] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/users/0ed95d70ce2a4f60b84d735219bd40e4 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:54.477836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.483992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.506178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412a418-f3ae-4e56-bc81-32841a65d1cc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.536922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.547145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.549007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.549530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.550025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.551672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.554941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.559240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 147/290] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:54.566778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.593368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687240c7-24a3-41e9-a5ec-37ab54c7c5d6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.633754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.635075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.636240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.636650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.691912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc774ac-7696-42c9-bbcb-481eb37160d3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:54.737817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:54.738981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:54.739341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:54.739687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/211f5fb7e54c4a1687f434d798dfd4f7/roles/288574a645fb48129c87eb8509d301e7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:54.742932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, user_id=211f5fb7e54c4a1687f434d798dfd4f7, role_id=288574a645fb48129c87eb8509d301e7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.744777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.761500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 148/291] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:13:54 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/users/211f5fb7e54c4a1687f434d798dfd4f7/roles/288574a645fb48129c87eb8509d301e7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:13:54.800908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:54.802400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:54.876787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:54.946238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 144/292] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:13:54.952387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:54.969496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:55.087279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010dc99-5e4d-4eca-ab55-4e4c05a7a5bb None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:55.126793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:55.127675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:55.127961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:55.128244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:55.129454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:55.130968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9dba3-2a6c-4edb-9c01-c0b0c67bf4fd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:55.133402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 149/293] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:54 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:55.140333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:55.269194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603436b-a58c-4596-ab2c-170637ebc471 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:55.307469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:55.308134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:55.308416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:55.308690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:55.309555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=0ed95d70ce2a4f60b84d735219bd40e4) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:55.309868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:55.311304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c21f5-d1c3-4876-8a2a-48f4de15f02c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:55.317579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 145/294] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:55 2022] GET /identity/v3/role_assignments?user.id=0ed95d70ce2a4f60b84d735219bd40e4&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:56.378614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5bd8-c1d2-4821-b6e3-5d72986b5baf None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.379157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5bd8-c1d2-4821-b6e3-5d72986b5baf None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.379233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d5bd8-c1d2-4821-b6e3-5d72986b5baf None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.380574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 150/295] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:56.385818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaabdd-23aa-4b22-9d78-4eecfd8e1f95 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.386204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaabdd-23aa-4b22-9d78-4eecfd8e1f95 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.386379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaabdd-23aa-4b22-9d78-4eecfd8e1f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.408347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2aaaabdd-23aa-4b22-9d78-4eecfd8e1f95 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:56.409560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2aaaabdd-23aa-4b22-9d78-4eecfd8e1f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:56.421581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:56.479914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 146/296] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:56.502750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5bd8-c1d2-4821-b6e3-5d72986b5baf None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:56.592787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:56.621716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d5bd8-c1d2-4821-b6e3-5d72986b5baf None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:56.628197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcace69-75e9-4157-bebf-145234aac379 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.628494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcace69-75e9-4157-bebf-145234aac379 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.628920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcace69-75e9-4157-bebf-145234aac379 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.630502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 147/297] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:56.636389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850f458-63a7-42e4-80ab-837b66b174b7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.636686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850f458-63a7-42e4-80ab-837b66b174b7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.637024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850f458-63a7-42e4-80ab-837b66b174b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.657406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2850f458-63a7-42e4-80ab-837b66b174b7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:56.658638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2850f458-63a7-42e4-80ab-837b66b174b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:56.659968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:56.660801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.661112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.661391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.677261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:56.685294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:56.686181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:56.687390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 151/298] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:56.693018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:56.743775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 148/299] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:56.745457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacfc8f-41ed-435c-af72-ae572ac85785 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.745889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacfc8f-41ed-435c-af72-ae572ac85785 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.746372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacfc8f-41ed-435c-af72-ae572ac85785 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.747818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 149/300] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:56.753705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.754196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.754676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.772044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:56.773869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:56.815166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dadf2-7e20-49d3-8436-49e774af7e28 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:56.852912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 150/301] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:56.852912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:56.858136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:56.858896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:56.859256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:56.859618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:56.860458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:56.860864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:56.862766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:56.863313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:56.866140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:56.879077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 152/302] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:56.879929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:56.964468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fa9a8-71c6-4944-92f9-80299b8b10f7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:56.980674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261982e-30b8-45e8-b8ac-524286486b68 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.001708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.002605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.002949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.003230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.014090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.014493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:57.014888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.015178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.015457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.016629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.018107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.019743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 153/303] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:57.024466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:57.024543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:57.025902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:57.026670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 151/304] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:13:57.029688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.094956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.095151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.132708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e0fe7-30ed-495e-9a3b-6ab211ffbcca None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.136768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382bc7a3-bfa2-49c2-9946-2f3fb089369c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.169546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.170428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.170722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.171009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] PATH_INFO: `/v3/users/45b38e70f6f0400394173e71c5784748` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.171296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.171960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.172247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.172527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] PATH_INFO: `/v3/groups` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.173250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.174993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.186264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] RBAC: Authorizing `identity:get_user(user_id=45b38e70f6f0400394173e71c5784748)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.187716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.189473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 154/305] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/users/45b38e70f6f0400394173e71c5784748 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:57.194807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 152/306] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Jun 16 00:13:57 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:13:57.194939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.197863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.330839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c226b9-e76b-4c36-a222-774d96460925 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.331696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c5dc4-8950-434a-8785-572390137fcb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.357390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.360786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.371195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.371701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.371887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.372156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.372385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.372551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.372672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.372975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.373320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, user.id=45b38e70f6f0400394173e71c5784748, scope.system=all) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:57.373636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.373900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:57.374016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.375091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee76932-718a-4985-9f7c-45e80b693088 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.375358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.385542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 153/307] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/role_assignments?user.id=45b38e70f6f0400394173e71c5784748&scope.system=all&role.id=288574a645fb48129c87eb8509d301e7 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:57.385542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f711-b342-45b5-b411-8f940e5b5153 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.385542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f711-b342-45b5-b411-8f940e5b5153 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.385542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20f711-b342-45b5-b411-8f940e5b5153 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.385542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 154/308] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:57.390796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 155/309] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:57.396483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.396779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.397122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.399085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.416734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:57.418822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:57.517470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 155/310] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:57.533169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216901ee-416f-48de-9915-a7d1ba3e0442 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.542646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.574451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.575469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.576046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.576705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] PATH_INFO: `/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.578193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe0d4ae4a64b3a904d657ad136b76e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.579891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.582346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 156/311] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:57.588117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.658938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d88104-b769-4486-a7dc-822df922da13 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.664832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.685017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:57.714897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.716009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.716303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.716585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.725509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae78f5-28a1-4dd8-86f1-f6766abaa031 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.736034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:57.747132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:57.748016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:13:57.748821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 156/312] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:57.758580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:57.769685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.770560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.770848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.771135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] PATH_INFO: `/v3/system/users/8dbe0d4ae4a64b3a904d657ad136b76e/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.772637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8dbe0d4ae4a64b3a904d657ad136b76e, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.775340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.785855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 157/313] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Jun 16 00:13:57 2022] PUT /identity/v3/system/users/8dbe0d4ae4a64b3a904d657ad136b76e/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:13:57.881529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474edac6-f9c4-485f-8e9e-9d287095b977 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:57.923522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:57.924128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:57.924417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:57.924696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:57.925506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:57.925869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:57.927297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:57.937840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 157/314] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:57.943077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:58.053871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaa6fc-8437-46c0-b80f-a8e19e993d63 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:58.073433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:58.092317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:58.093254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:58.093746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:58.094071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] PATH_INFO: `/v3/users/211f5fb7e54c4a1687f434d798dfd4f7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:58.095401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] RBAC: Authorizing `identity:get_user(user_id=211f5fb7e54c4a1687f434d798dfd4f7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:58.097203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:58.102823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 158/315] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:13:57 2022] GET /identity/v3/users/211f5fb7e54c4a1687f434d798dfd4f7 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:58.106122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:58.155935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:58.218935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38cbe4-685f-4202-839a-9d9562e1f3fb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:58.259541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:58.260196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:58.260484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:58.260763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:58.332660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:58.334431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:58.379672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:58.444460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 158/316] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:13:58.452562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:58.570616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2896d0-b5d3-4ec4-ad5c-1ba90eac10fa None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:58.610202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:58.611078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:58.611367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:58.611646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:58.612838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:58.614303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746145db-0dce-40b3-8096-e61e231ec5ab None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:58.616325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 159/317] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:13:58 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:58.622756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:58.728917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6899b-aa34-4bdf-b56b-87ee0e0e90eb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:58.763478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:58.764137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:58.764468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:58.764753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:58.765613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, scope.project.id=fb75e39589714c508c3d9f41dd28c429, user.id=211f5fb7e54c4a1687f434d798dfd4f7) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:13:58.765926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:58.767348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c8b61-10e6-4536-a98d-b95fd4214955 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:58.774304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 159/318] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:13:58 2022] GET /identity/v3/role_assignments?user.id=211f5fb7e54c4a1687f434d798dfd4f7&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=288574a645fb48129c87eb8509d301e7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:13:59.549773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b64fc-5e5b-47b7-a5d8-a9e60853a51d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.550310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b64fc-5e5b-47b7-a5d8-a9e60853a51d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.550383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b64fc-5e5b-47b7-a5d8-a9e60853a51d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.552080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 160/319] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:13:59.558526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.559028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.559098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.571110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:59.577223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:59.578429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:59.598924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d24b8-7c04-4db1-a100-5697016b99e6 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.599219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d24b8-7c04-4db1-a100-5697016b99e6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.599672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d24b8-7c04-4db1-a100-5697016b99e6 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.601625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 161/320] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:13:59.606544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.606835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.607277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.624672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:13:59.625910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:13:59.655916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 160/321] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:13:59.678072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:59.678740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:59.697085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 162/322] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:13:59.719048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:59.787153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191532a0-bbb2-41ac-b6f2-fcdd1f88f0ea None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:59.826515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:59.827311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.827652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.827932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.829126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:13:59.830811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:13:59.837073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 161/323] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:13:59.837221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4cfc3-0174-4cbd-888b-d0215e99a6bd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:59.838864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:59.841763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:59.873125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:13:59.874039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:13:59.874440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:13:59.874718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:13:59.887271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:13:59.894959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:13:59.895888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:13:59.896687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 163/324] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:13:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:13:59.902548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:13:59.959684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5b1e4-55b3-46f7-9f9f-5088857fea8a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:13:59.961568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:13:59.999966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.001050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.001485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.001771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] PATH_INFO: `/v3/groups` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.002473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.003919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18977645-37d5-476e-9da4-0dc2e3ca977d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.017836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 162/325] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Jun 16 00:13:59 2022] POST /identity/v3/groups => generated 226 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:14:00.018085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880a247-db9a-4485-b47b-23cf95642b6c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.018946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1bb0d-1b58-4764-ab02-15c1a4174152 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.019243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1bb0d-1b58-4764-ab02-15c1a4174152 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.019527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1bb0d-1b58-4764-ab02-15c1a4174152 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.020793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 163/326] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:00.027299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.027598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.027888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.045682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:00.046945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:00.058745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.059434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.059776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.060063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.060830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:00.061173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.062589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.071576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 164/327] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:13:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:00.077370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.115718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:00.128742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 164/328] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:00.152634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.196467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874fed7-f574-4ad9-a70c-50e417783b46 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.205384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:00.238747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.239415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.239524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.239771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] PATH_INFO: `/v3/users/45b38e70f6f0400394173e71c5784748` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.242223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] RBAC: Authorizing `identity:get_user(user_id=45b38e70f6f0400394173e71c5784748)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.244485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.247383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 165/329] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:14:00 2022] GET /identity/v3/users/45b38e70f6f0400394173e71c5784748 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:00.253351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.289327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d39f-82b1-4cd7-bd4b-21453d514443 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.336526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.337322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.337610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.337892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.350712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:00.359380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:00.360268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:00.361190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 165/330] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:00.366260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.388560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b16244-dde6-47bf-b0cf-4e7c5a857433 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.432479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.433391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:00.434274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.434813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.435292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] PATH_INFO: `/v3/system/users/45b38e70f6f0400394173e71c5784748/roles/288574a645fb48129c87eb8509d301e7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.437494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=45b38e70f6f0400394173e71c5784748, role_id=288574a645fb48129c87eb8509d301e7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.439605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.450605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 166/331] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Jun 16 00:14:00 2022] PUT /identity/v3/system/users/45b38e70f6f0400394173e71c5784748/roles/288574a645fb48129c87eb8509d301e7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:14:00.496682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb22b73-89d3-4b41-8c23-7310d7d76d32 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.539802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.540249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.540498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.540828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.541633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:00.541952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.543349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.555585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 166/332] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:00.560715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.621635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:00.698024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8f65d-a427-43d6-ab00-a85780dff7f4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.743119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.743846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.744132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.744407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] PATH_INFO: `/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.745665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] RBAC: Authorizing `identity:get_user(user_id=8dbe0d4ae4a64b3a904d657ad136b76e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.747386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e08c6-421c-4824-bc19-66f9916b9b9a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.749345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 167/333] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:14:00 2022] GET /identity/v3/users/8dbe0d4ae4a64b3a904d657ad136b76e => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:00.758684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:00.869957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7414d-9285-493b-8aa7-5eda534436fc None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:00.889352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:00.912279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:00.912976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:00.913264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:00.913643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:00.914930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] RBAC: Adding query filter params (role.id=f5a6a04dca1e4234b7616969cfcd7643, user.id=8dbe0d4ae4a64b3a904d657ad136b76e, scope.system=all) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:00.915251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:00.916703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eae9c0-7734-46fa-ab00-774fdc832e32 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:00.932121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 167/334] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Jun 16 00:14:00 2022] GET /identity/v3/role_assignments?user.id=8dbe0d4ae4a64b3a904d657ad136b76e&scope.system=all&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:02.728439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb3f-5cc8-456b-abc0-026f62057152 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:02.728935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb3f-5cc8-456b-abc0-026f62057152 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:02.729019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb3f-5cc8-456b-abc0-026f62057152 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:02.730406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 168/335] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:02.735712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:02.736133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:02.736273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:02.761024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:02.762506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:02.849444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 168/336] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:02.875331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eb3f-5cc8-456b-abc0-026f62057152 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.002668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eb3f-5cc8-456b-abc0-026f62057152 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.010965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:03.048376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:03.049223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.049679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.049973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.064954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-e83baa57-b383-4f0f-8312-3f60eb1d60e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:03.073422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:03.074337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:03.075197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:03.078237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 169/337] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:03.084132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.196631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed090970-52cc-4b9b-a971-422076639589 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.197370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed090970-52cc-4b9b-a971-422076639589 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.197965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed090970-52cc-4b9b-a971-422076639589 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.199556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 170/338] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:03.206313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.206892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.207425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.213600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208d21b-ca7d-46ac-a966-e07765cb78c2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.216429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:03.245922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:03.247505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:03.253283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:03.253900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.254186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.254464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.255220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:03.255529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:03.256915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:03.268502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 169/339] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:03.274911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.378230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d16e69-57f0-4345-8202-10ede8832791 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.423222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:03.423901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.424189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.424472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.450842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 171/340] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:03.485268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:03.495557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.571589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:03.573133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:03.687542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:03.708895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e886e7-2fdf-4823-9884-ef15df006404 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.740307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 170/341] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:03.747170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.778577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:03.779413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.779701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.779983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.806400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:03.821268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:03.822296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:03.823122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 172/342] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:03.828597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:03.849660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:03.907669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db07ab-6c95-4614-807e-9d4159fca0f7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.979892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:03.980774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.981092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.981363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.982820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:03.984425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17283b-12c5-4aed-b0d5-00a1a57affb0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:03.986834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 171/343] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:03.986970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5ef-f558-40c2-87e5-b6cfabb6cc8a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:03.988423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78c4-2b21-4d37-a0e0-f4a18a4474b5 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:03.989035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78c4-2b21-4d37-a0e0-f4a18a4474b5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:03.989454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78c4-2b21-4d37-a0e0-f4a18a4474b5 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:03.996795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 172/344] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:03.997885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb78c4-2b21-4d37-a0e0-f4a18a4474b5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.052475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.053192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.053479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.053758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.054867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:04.055189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:04.056789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13886b-881a-400e-83f8-6b1b705b732b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:04.076244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 173/345] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:04.077765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.078063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.078345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.109967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:04.111262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:04.126721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb78c4-2b21-4d37-a0e0-f4a18a4474b5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:04.137768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.170293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.171164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.171449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.171725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.174159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.181091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:04.182605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:04.184665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 173/346] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:03 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:04.185273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.250026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 174/347] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:04.250694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.302353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c93470-45b8-49c2-90f3-05360bb57984 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:04.339572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.340163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.340455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.340739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.399820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fbf0-31cb-47dc-ae97-201792c359ed None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:04.431580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.438462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:04.440769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f425f-85c2-4edb-95dd-4eb9b6c7cc6e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:04.453432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.454126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.454408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.454995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.456356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:04.456676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:04.458923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:04.468210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 175/348] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:04.469131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.504356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.620520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 174/349] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:04.620520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f82df-f481-4304-90ad-7d9959da4e74 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.620520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f82df-f481-4304-90ad-7d9959da4e74 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.620520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f82df-f481-4304-90ad-7d9959da4e74 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.620520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 175/350] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:04.622096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85102-6a2c-4ee3-a6e5-1418ee81918c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.622393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85102-6a2c-4ee3-a6e5-1418ee81918c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.622674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85102-6a2c-4ee3-a6e5-1418ee81918c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.629007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 176/351] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:04.630121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85102-6a2c-4ee3-a6e5-1418ee81918c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.654119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10b5de-c2bb-40d6-9cb9-92d56794e2c4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:04.705030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.706224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.706713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.707188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.721293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-11871783-089b-427a-8c7e-3a5819ede8a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:04.731243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:04.732206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:04.733045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:04.740860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 176/352] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:04.743309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.743607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.743892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.770555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:04.772116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:04.781460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.787044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85102-6a2c-4ee3-a6e5-1418ee81918c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:04.834618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:04.835612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.835955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.836240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.837477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:04.838915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6d39e-3755-4d7c-8d3e-ab4f3d7f9972 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:04.841224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 177/353] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:04.846181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:04.846476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:04.846757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:04.863566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 177/354] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:04.866477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.868411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:04.869783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:04.869962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:04.947504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 178/355] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:04.954142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:04.996257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f40a8c-bd1f-4471-8969-fcde78fdfe42 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.040371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.041088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.041622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.041622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.042719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:05.042719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.044071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.053413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 178/356] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:05.055135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.092482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.099632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87b62f-ce3b-4c39-a177-7f973fad6e31 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.141419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.148938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.149832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.150325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.150602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.151926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.153932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.155730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 179/357] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:04 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:05.156668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.187076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f0694-867c-4273-ae21-e4b8f1f7ee36 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.229970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.230703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.230995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.231286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.243429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-19169797-9489-4f96-8416-ab12443dcb52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:05.251227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:05.252126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:05.253350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 179/358] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:05.254378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513651-f2a9-4bed-b5a4-940600d51913 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.254588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513651-f2a9-4bed-b5a4-940600d51913 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.254869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513651-f2a9-4bed-b5a4-940600d51913 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.256139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 180/359] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:05.256823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513651-f2a9-4bed-b5a4-940600d51913 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.284801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa0f6e-5beb-4fa0-bac6-44a20ca86bf9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.337394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.337615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.338239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.338239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.352780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:05.361630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:05.362506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:05.363276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:05.370167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 180/360] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:05.370167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.397080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513651-f2a9-4bed-b5a4-940600d51913 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.402879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.431230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.442746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.443707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.444208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.445269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.510888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0053-842c-4c7f-b5d3-a103e28eb856 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.534421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.535887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff71060-dbac-4693-abc5-488aba2375c0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.558214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.559209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.559506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.559781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.560634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:05.560984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.562896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.571041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 181/361] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:05.571734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.701331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.706595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 181/362] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:05.708826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.709183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.709471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.724820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04ddc-0826-440e-a8ee-e7ba14d12dbf None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.742888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:05.744526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:05.775711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.776548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.776692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:05.776931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.777892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.778270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:05.778814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.781833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.791965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 182/363] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:05.795251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.850022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 182/364] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:05.851422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.916543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb39156-e366-44a5-9972-b1922c0ab791 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:05.952052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:05.952612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:05.952991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:05.953338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:05.954157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:05.954533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:05.955996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:05.968271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 183/365] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:05.969192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:05.995825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b05594-02c3-4563-8643-a3b26261aec7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.001460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.047689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.048532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.048866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.049185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.050433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.052167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.054682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 183/366] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.055357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.084305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.095839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204358e7-fc66-45b5-9903-463f6a9a0d9a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.135669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.136574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.136859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.137169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] PATH_INFO: `/v3/users/45b38e70f6f0400394173e71c5784748` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.138419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] RBAC: Authorizing `identity:get_user(user_id=45b38e70f6f0400394173e71c5784748)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.139866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.141552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 184/367] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:14:05 2022] GET /identity/v3/users/45b38e70f6f0400394173e71c5784748 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.142328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.170528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0303115-b03b-4e99-b2ed-abb2ab29a08e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.209977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.211376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.211649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.211978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.224432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:06.230252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:06.233873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:06.234789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:06.235560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 184/368] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:06.236222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.251507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401367d2-0c81-4d8b-9ad6-85d3b36c7e55 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.290498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.291169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.291460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.291745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.341316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.354439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c0dc2-98df-49bb-b36e-92a414c02b1e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.370922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.372413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.393456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.394336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.394667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.394961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.396989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.398667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.401120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 185/369] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.402025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.467160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.492032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 185/370] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:06.493195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.519463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a3e72-e2b1-4ed2-9400-f5a2c0f2bf36 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.559095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.559866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.560165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.560450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.561319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] RBAC: Adding query filter params (role.id=288574a645fb48129c87eb8509d301e7, user.id=45b38e70f6f0400394173e71c5784748, scope.system=all) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:06.561700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.563360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.579131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 186/371] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/role_assignments?user.id=45b38e70f6f0400394173e71c5784748&scope.system=all&role.id=288574a645fb48129c87eb8509d301e7 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.580346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.609021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85a0-2772-4902-b8a6-5801611718be None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.617471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.661633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.662335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.662625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.662906 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.664837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:06.665185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.666614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.685228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 186/372] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.686104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.697770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174dd71-ef27-4c43-80fb-f75805955814 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.739838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.739838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.740138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.740418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.742052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=fb75e39589714c508c3d9f41dd28c429) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:06.743001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:06.743981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:06.752854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 187/373] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:06.753840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:06.771187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.849101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:06.888235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b619e-c1b3-46bc-ac97-6802bcf9b262 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:06.930450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:06.931465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:06.931898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:06.932336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:06.962603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83adce9-85f6-46ff-9232-6f3d9cc7e82c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.008701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.010922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219d08b-faae-4e36-9707-bb17a3e5764a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.085746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.086724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.087012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.087288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.093846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.095329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.097172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 187/374] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.104993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.105668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:07.147906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 188/375] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:07.149906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59fee-cca4-403b-9f77-b583d2135840 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.150459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59fee-cca4-403b-9f77-b583d2135840 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.150743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e59fee-cca4-403b-9f77-b583d2135840 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.153180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 189/376] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:07.154080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e59fee-cca4-403b-9f77-b583d2135840 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.246272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6c406-8700-47ff-9176-d2c79de5de5d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.258598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:07.286530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.287764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.288792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.289289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.290723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.292384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892b485-62f1-45df-a297-ecd8e874ea25 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.294613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e59fee-cca4-403b-9f77-b583d2135840 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.296041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 188/377] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.297461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.297946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.298427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.321757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:07.323269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:07.343004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.344164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.344455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.344738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] PATH_INFO: `/v3/projects/3c21334330db408fb0ba251a2f04ab82` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.345960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] RBAC: Authorizing `identity:get_project(project_id=3c21334330db408fb0ba251a2f04ab82)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.347840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.350161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 190/378] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/projects/3c21334330db408fb0ba251a2f04ab82 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.352858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.402169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 189/379] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:07.407567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:07.409490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.478467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e16b10-06d5-4004-a390-b65daa050060 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.521623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.523216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.524330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.524818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.526143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=fb75e39589714c508c3d9f41dd28c429) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:07.526656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.528343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.540998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 191/380] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.542295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.546441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732b824-4c06-45e3-8cc0-5c5c2a4f3a52 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.561385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:07.598403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.599634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.599920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.600195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] PATH_INFO: `/v3/groups/e6615e64e1e74047a7ed030299a9c8b2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.607321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] RBAC: Authorizing `identity:get_group(group_id=e6615e64e1e74047a7ed030299a9c8b2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.608818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.610683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 190/381] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/groups/e6615e64e1e74047a7ed030299a9c8b2 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.617927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.760785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4d34-6188-4b2a-ba07-4a57023b3f6e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.763518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:07.778787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f81b-5c1e-43f6-8f8c-d33fbda964c9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:07.826214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.826896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.827176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.827449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.828345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] RBAC: Adding query filter params (group.id=e6615e64e1e74047a7ed030299a9c8b2, role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=3c21334330db408fb0ba251a2f04ab82) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:07.828658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:07.830222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:07.835279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:07.836124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:07.836405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:07.836783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:07.838004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 191/382] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/role_assignments?group.id=e6615e64e1e74047a7ed030299a9c8b2&scope.project.id=3c21334330db408fb0ba251a2f04ab82&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:07.856564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:07.864646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:07.865820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:07.868144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 192/383] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:07.873467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:07.942198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:08.069091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99068176-83c8-4988-acd6-06c7ca5b38ca None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.154979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.156050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.156555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.157056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.158022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:08.158549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:08.160117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc10f1-d679-4981-a085-fca486024889 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:08.176490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 192/384] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:08.182779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:08.353018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec44a1c-4056-4d39-9f9d-02fd013aea9f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.358325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761922-539a-4a24-a472-146e2b063fd0 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.358864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761922-539a-4a24-a472-146e2b063fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.359366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5761922-539a-4a24-a472-146e2b063fd0 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.360943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 193/385] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:08.373283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.373886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.374402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.393273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.394142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.394425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.394698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.395873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:08.397350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:08.404346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 193/386] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:08 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:08.407250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:08.412023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:08.414243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:08.450126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:08.538036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 194/387] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:08.563869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:08.597018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d066d2-e915-4b6d-b15d-6d0653b53e66 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.613297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:08.661613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.662513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.662806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.663598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.665292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:08.666776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:08.668816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 194/388] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:08 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:08.674199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:08.696675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75901a7-95c0-4af1-ba38-1c992713e0cf None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.737279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.738161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.738445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.738771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.750827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:08.759934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:08.760795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:08.761613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 195/389] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:08.768091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:08.826130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:08.831057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50866f-8b4c-49a7-b63a-e1febce040a5 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.878733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:08.886647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd9e0-01ba-483e-a8f1-0a8a0bf329fd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:08.889776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.890692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.890980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.891314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/groups/70e461f3bdd541f2a90e9199745072dd/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.894070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, group_id=70e461f3bdd541f2a90e9199745072dd, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:08.895540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:08.927552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 195/390] 10.222.0.23 () {58 vars in 1442 bytes} [Thu Jun 16 00:14:08 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/groups/70e461f3bdd541f2a90e9199745072dd/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:14:08.943781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:08.945536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:08.945972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:08.946393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:08.947298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:08.947786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:08.949387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:08.958872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 196/391] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:08.965378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:09.090571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bccde1-2556-4f93-bf3c-0f97b6f425bc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:09.136746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:09.137943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:09.138391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:09.138828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:09.140233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:09.141921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:09.144120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 196/392] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:08 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:09.150714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:09.261607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437c595-10c5-46b5-88c1-efeb82bf5cf8 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:09.302847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:09.304248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:09.304537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:09.304813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:09.306038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:09.307480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecadd1-b2b4-4acf-88f2-aed62151648d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:09.309583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 197/393] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:09 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:09.315099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:09.365906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:09.457839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e945a6e-5027-42f2-a21b-90b8e0fff6fa None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:09.500751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:09.502103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:09.502689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:09.503181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/groups/70e461f3bdd541f2a90e9199745072dd/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:09.505663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbbcea3cbf41609874338eb93f9be7, group_id=70e461f3bdd541f2a90e9199745072dd, role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:09.508026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31cea-1991-42fe-9958-5ab351d745aa None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:09.527251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 197/394] 10.222.0.23 () {58 vars in 1442 bytes} [Thu Jun 16 00:14:09 2022] PUT /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7/groups/70e461f3bdd541f2a90e9199745072dd/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:09.898284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080847-b885-4f76-9202-bac6dc7b9c03 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:09.898567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080847-b885-4f76-9202-bac6dc7b9c03 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:09.899212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3080847-b885-4f76-9202-bac6dc7b9c03 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:09.901088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 198/395] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:09.907254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:09.907564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:09.907847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:09.926407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:09.927597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:10.018584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 198/396] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:10.053595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3080847-b885-4f76-9202-bac6dc7b9c03 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.079965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.181104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3080847-b885-4f76-9202-bac6dc7b9c03 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:10.230324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:10.231134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.231418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.231698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.246908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5873c83f-7f5e-432d-b257-1635ac7987f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:10.254909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:10.255792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:10.256591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 199/397] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:10.262727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.302213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.435452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c28480-3312-4f24-9cf9-4b0a0384a75e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:10.472432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30288-e098-4eb0-8c4a-8657e5bb41d1 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.472754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30288-e098-4eb0-8c4a-8657e5bb41d1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.473023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30288-e098-4eb0-8c4a-8657e5bb41d1 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.474456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 200/398] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:10.479889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.480552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.481081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.503501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:10.504541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.505079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.505583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.506547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:10.508418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:10.508983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:10.510577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:10.512042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:10.527763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 199/399] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:10.533929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.630738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.633208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 201/400] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:10.646162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c07a6a-8086-4f56-bb7b-2fed3743b2e4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:10.665213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.677227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.701627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:10.702525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.702818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.703103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.704340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:10.705832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:10.707502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 200/401] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:10.724716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.794013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16644e3-0286-4115-95e9-650f2dcc8afa None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:10.830343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:10.831491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.831971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.832465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.843451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-ea8c33ba-a672-48c6-bd17-a1e6170862c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:10.852444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:10.853384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:10.854153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:10.855291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d057ca5-dc4d-421a-8ef1-24ac68cbeedf None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:10.856666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 202/402] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:10.865202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa2979-b444-4056-be14-9fae3d3e256e None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.865668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa2979-b444-4056-be14-9fae3d3e256e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.866076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa2979-b444-4056-be14-9fae3d3e256e None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.868502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 203/403] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:10.873818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.874283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.874709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.892966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:10.894323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:10.896260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:10.897407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:10.898104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:10.898104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:10.899301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:10.901043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:10.903052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 201/404] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:10.903834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:10.923419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.978792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:10.988970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 204/405] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:10.989870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.027096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b261db-2d3b-41c8-b651-8ef8a05ce0af None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.065959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.067068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.067552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.068069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.069061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:11.070143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.072189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.081873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 202/406] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:11.083161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.106220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea30811-02b6-44c8-a463-347471fafb18 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.152504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.153654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.154083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.154522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/groups/70e461f3bdd541f2a90e9199745072dd/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.156683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, group_id=70e461f3bdd541f2a90e9199745072dd, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.158321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.171770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.178050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 205/407] 10.222.0.23 () {58 vars in 1442 bytes} [Thu Jun 16 00:14:10 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/groups/70e461f3bdd541f2a90e9199745072dd/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:11.179770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.212298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce571f-0eef-4871-9b3a-d1eca6ba8819 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.253286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.299233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbbe14-c210-44f5-9815-38d47876b85e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.317663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.318718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.318979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.319305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.341073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.342439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.342972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.343534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.345144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.346874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.349221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 206/408] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:11.358727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.358905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:11.367889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:11.368838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:11.369628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:11.373293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 203/409] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:11.386942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.486217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cd9e2-dd3f-488e-b984-3210e7706cae None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.524625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3939b-e847-433f-b22f-4ab0bde0474f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.537732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.557596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.558481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.558751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.559067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.560290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.561683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.563977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 207/410] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:11.569098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.571187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.572424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.574701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.575196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.576200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:11.577955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.579587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.597311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.605386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 204/411] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:11.607537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.723231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d6d08-5f2e-4b01-a933-d834790ea5ac None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.738591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc9b4-3936-4dbf-b863-cb92594d2fe8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.768237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.769450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.770295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.770589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429/groups/70e461f3bdd541f2a90e9199745072dd/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.773404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] RBAC: Authorizing `identity:create_grant(project_id=fb75e39589714c508c3d9f41dd28c429, group_id=70e461f3bdd541f2a90e9199745072dd, role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.774720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.774970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b6c9f-dbed-400d-a7ec-da7c4458d67c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.776145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.776581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.777040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] PATH_INFO: `/v3/groups/e6615e64e1e74047a7ed030299a9c8b2` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.778362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] RBAC: Authorizing `identity:get_group(group_id=e6615e64e1e74047a7ed030299a9c8b2)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.779965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68efd9f-2e79-4b5d-9c97-1a290921c361 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:11.782177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 205/412] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/groups/e6615e64e1e74047a7ed030299a9c8b2 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:11.783466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb4e4-dc90-42fa-84fa-2c33055351d3 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.783762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb4e4-dc90-42fa-84fa-2c33055351d3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.784044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb4e4-dc90-42fa-84fa-2c33055351d3 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.789386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 206/413] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:11.791279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffb4e4-dc90-42fa-84fa-2c33055351d3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:11.794126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 208/414] 10.222.0.23 () {58 vars in 1442 bytes} [Thu Jun 16 00:14:11 2022] PUT /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429/groups/70e461f3bdd541f2a90e9199745072dd/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:14:11.797015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.797609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.798183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.818418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:11.820333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:11.901171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.938931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:11.945523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffb4e4-dc90-42fa-84fa-2c33055351d3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:11.984263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 209/415] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:11.991993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:11.993263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:11.993742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:11.994232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] PATH_INFO: `/v3/projects/3c21334330db408fb0ba251a2f04ab82` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:11.995741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] RBAC: Authorizing `identity:get_project(project_id=3c21334330db408fb0ba251a2f04ab82)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:11.997624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:12.000003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 207/416] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:11 2022] GET /identity/v3/projects/3c21334330db408fb0ba251a2f04ab82 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:12.005608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:12.017061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:12.126843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06b355-4af2-41c9-9a83-6452261d15b9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:12.164778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:12.168905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.168905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.168905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] PATH_INFO: `/v3/projects/3c21334330db408fb0ba251a2f04ab82/groups/e6615e64e1e74047a7ed030299a9c8b2/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.168905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c21334330db408fb0ba251a2f04ab82, group_id=e6615e64e1e74047a7ed030299a9c8b2, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:12.170182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:12.177256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84653cd1-9506-4988-ab21-8c86c1f7cc8c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:12.189394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 210/417] 10.222.0.23 () {58 vars in 1442 bytes} [Thu Jun 16 00:14:12 2022] PUT /identity/v3/projects/3c21334330db408fb0ba251a2f04ab82/groups/e6615e64e1e74047a7ed030299a9c8b2/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:14:12.247957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:12.248733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.249053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.249341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.256044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:12.267176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-43e25f77-2c98-4f4e-9c6a-4b5c3674a066 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:12.276570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:12.277559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:12.278355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:12.283054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 208/418] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:12.284315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:12.346988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673a84a-7e75-4880-9b79-c4f33e3c4d01 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.347589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673a84a-7e75-4880-9b79-c4f33e3c4d01 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.348053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673a84a-7e75-4880-9b79-c4f33e3c4d01 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.349946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 209/419] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:12.363328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.363895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.364381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.365027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:12.386013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:12.387828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:12.448672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0bd2-2e2a-46af-af06-0cfd970d4386 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:12.491164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 210/420] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:12.499666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:12.501065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.501567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.502043 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.503019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] RBAC: Adding query filter params (name=member) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:12.503569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:12.505258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:12.520965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:12.528053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 211/421] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:12.528975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:12.672533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27c744-59f9-4e20-8385-39f35e2c8763 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:12.680155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:12.714805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:12.715478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.715775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.716056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.735383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645bf4c-0c42-4414-b91d-146394b73b6a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:12.788873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:12.809045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:12.810210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:12.810725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:12.811208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:12.813027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:12.815863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:12.837139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:12.846907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:12.847799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:12.848579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:12.851961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 211/422] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:12.857669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.007713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 212/423] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:13.013218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.025439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.040596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4d2f3-0110-4930-bc0b-278fc0d19832 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.108085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.109265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.109751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.110254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.111250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:13.111786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.114420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.131075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 212/424] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:13.138555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.198273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.232185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271811-6f98-4750-af68-edc384805e7f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.250139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a614e-ed07-4cd1-84a0-a0d16e3e41bc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.286641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.287661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.288722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.289391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.302526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.303325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.303743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.304015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.309860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.311092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.312839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 213/425] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:13.318153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.360616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.362618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.417276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.471287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 213/426] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:13.477816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.489230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.583250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0b64-3b0e-4558-94a5-7469bf103481 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.607424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737c14-4a8c-4b45-941b-8a9c60b094fe None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.645673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.648423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.652416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.652775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] PATH_INFO: `/v3/projects/1bdbbcea3cbf41609874338eb93f9be7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.655148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbbcea3cbf41609874338eb93f9be7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.656676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.666139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 214/427] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/projects/1bdbbcea3cbf41609874338eb93f9be7 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:13.674373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.685022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.686453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.686959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.687508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.689065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.690949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.693382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.693616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 214/428] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:13.702803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.798641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ecea7-8bd2-429b-870b-024671dba6ce None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.845994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.847410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.847845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.848409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.849927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.852787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.854894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 215/429] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:13.861172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:13.912189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768df5ee-4db5-4832-aeed-ec7c1bce6a88 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:13.951692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:13.952632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.953090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.953531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.954513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:13.954991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:13.956604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b619a-1d0e-443a-9b66-f5b89a901121 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:13.964476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 215/430] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:13.966103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e9f6-6eff-4f20-9924-5c8297682855 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.966614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e9f6-6eff-4f20-9924-5c8297682855 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.967106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e9f6-6eff-4f20-9924-5c8297682855 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.968590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 216/431] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:13.973339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:13.976329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4fa6e-a0d0-4fa3-b274-a96a1ef0f7db None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:13.977146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4fa6e-a0d0-4fa3-b274-a96a1ef0f7db None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:13.979061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4fa6e-a0d0-4fa3-b274-a96a1ef0f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:13.999290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.004579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e869c3-f9e3-456e-b3a8-d7ca8073cb60 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.019202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-90f4fa6e-a0d0-4fa3-b274-a96a1ef0f7db None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:14.020823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-90f4fa6e-a0d0-4fa3-b274-a96a1ef0f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:14.059108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.060195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.061860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.062361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.066333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=1bdbbcea3cbf41609874338eb93f9be7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:14.066986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:14.068703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:14.082664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 216/432] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:13 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=1bdbbcea3cbf41609874338eb93f9be7&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:14.136032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 217/433] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:14.171720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.208957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968037c-98bf-422d-873b-1e03484efe3e None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.209624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968037c-98bf-422d-873b-1e03484efe3e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.210645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968037c-98bf-422d-873b-1e03484efe3e None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.212700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 218/434] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:14.219890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.220188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.220472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.245621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:14.247075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:14.328469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745101a-f87f-40a0-9e58-d450006d1d30 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.342485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 219/435] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:14.367063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.372041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.373165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.373678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.374145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.387911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:14.399318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:14.400491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 00:14:14.401380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:14.401980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 217/436] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:14.407769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.422547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.459438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.502745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ea755-d929-410e-a7cb-caba0aecc0dd None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.548848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.549708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.549995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.550151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b81fe9-6eb2-4b4b-9931-9d5c57405efc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.550553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.562053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:14.571622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:14.572723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:14.573709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:14.576172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 220/437] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:14.581870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.590761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.592186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.592652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.593139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.594401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] RBAC: Adding query filter params (name=member) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:14.595201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:14.596953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:14.607826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 218/438] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:14.613478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.701035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b443c8f-5461-4bd6-a8a8-8e0c5ec41da3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.701870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.739323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.742542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8e440-0514-4fee-9202-c07651e096ae None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.754045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.754999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.755431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.755846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.757391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:14.757879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:14.759426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:14.771494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 221/439] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:14.778089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.791853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.792963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.793466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.793966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.872438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:14.873949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ca345-9912-47b0-aca4-766d678bc5d3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:14.887440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ea292-09a0-4ac6-82c1-fc1ee1fb2c93 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:14.920498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:14.921168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.921455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.921733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.940939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.971936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 219/440] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:14:14.974288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451607b-3ba7-4cd1-bd96-cee9e25d55af None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:14.974587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451607b-3ba7-4cd1-bd96-cee9e25d55af None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:14.974875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451607b-3ba7-4cd1-bd96-cee9e25d55af None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:14.976420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 220/441] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:14.977303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451607b-3ba7-4cd1-bd96-cee9e25d55af None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:14.978180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:14.991952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:14.993564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6162b-6b53-42e2-abc9-8c552774c3c5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.077687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451607b-3ba7-4cd1-bd96-cee9e25d55af None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.091667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 222/442] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:15.092959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.093257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.093542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.109313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:15.110506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:15.112266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.113157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.113443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.113717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.114875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.116828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.118531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 221/443] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:14 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.118531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.162793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.177672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 223/444] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:15.178990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.197966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.231426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebf23a-f2b9-41d7-ac1c-1d271e386a22 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.264997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.265865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.266157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.266430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] PATH_INFO: `/v3/projects/fb75e39589714c508c3d9f41dd28c429` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.267901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] RBAC: Authorizing `identity:get_project(project_id=fb75e39589714c508c3d9f41dd28c429)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.269378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.271066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 222/445] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/projects/fb75e39589714c508c3d9f41dd28c429 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.271834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.282436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0271b-f830-4860-b76c-369689e01c34 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.316407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.317294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.317576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.317855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.319059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.320705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.322346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 224/446] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.323099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.369164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933213e-5585-45a9-a4f4-b0e8abdf2409 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.376221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.403960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.404752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.405067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.405346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.407603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.415987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:15.422904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01dff9-2be5-4bb2-b2a8-ff164815629a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:15.423586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:15.424560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:15.425390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 223/447] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:15.426046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.466590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.467519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.467806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.468086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] PATH_INFO: `/v3/groups/70e461f3bdd541f2a90e9199745072dd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.469996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] RBAC: Authorizing `identity:get_group(group_id=70e461f3bdd541f2a90e9199745072dd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.472124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.473790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 225/448] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/groups/70e461f3bdd541f2a90e9199745072dd => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.474552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.531213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca780e-dbdc-45c4-986b-cfc8097118d1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.563155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.563814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.564106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.564396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.565268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=f5a6a04dca1e4234b7616969cfcd7643, scope.project.id=fb75e39589714c508c3d9f41dd28c429) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:15.565589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.567068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.574089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 224/449] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=f5a6a04dca1e4234b7616969cfcd7643 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.574878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.576404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07648-a463-420d-8739-0871bc656702 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.617018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe99fae-f893-469b-b759-638be384677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.617804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99fae-f893-469b-b759-638be384677a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.618214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99fae-f893-469b-b759-638be384677a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.618511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99fae-f893-469b-b759-638be384677a None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.619286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99fae-f893-469b-b759-638be384677a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:15.619609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99fae-f893-469b-b759-638be384677a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.620034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.621042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99fae-f893-469b-b759-638be384677a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.633629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 226/450] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.639625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe99fae-f893-469b-b759-638be384677a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:15.660383 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:15.701433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9690-45f0-4dd6-b17c-d68eedd248af None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.760296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.760998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.761291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.761570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.762417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] RBAC: Adding query filter params (group.id=70e461f3bdd541f2a90e9199745072dd, role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=fb75e39589714c508c3d9f41dd28c429) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:15.762732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.764183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.771033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 225/451] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/role_assignments?group.id=70e461f3bdd541f2a90e9199745072dd&scope.project.id=fb75e39589714c508c3d9f41dd28c429&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:15.778660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe99fae-f893-469b-b759-638be384677a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:15.819977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:15.820648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:15.820968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:15.821252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:15.903627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:15.905577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:15.926778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:16.017166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 227/452] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:16.023119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:16.127715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:16.139513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63716bf0-8d1d-478c-94f6-8133c3ecfc50 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:16.174871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:16.175747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:16.176041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:16.176325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] PATH_INFO: `/v3/projects/3c21334330db408fb0ba251a2f04ab82` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:16.177595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] RBAC: Authorizing `identity:get_project(project_id=3c21334330db408fb0ba251a2f04ab82)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:16.179067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:16.180760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 226/453] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:14:16 2022] GET /identity/v3/projects/3c21334330db408fb0ba251a2f04ab82 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:16.186248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:16.285600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126802ef-d590-4339-a3c0-ccd319c376cc None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:16.301291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:16.318983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:16.320796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:16.321109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:16.321388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] PATH_INFO: `/v3/groups/e6615e64e1e74047a7ed030299a9c8b2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:16.322554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] RBAC: Authorizing `identity:get_group(group_id=e6615e64e1e74047a7ed030299a9c8b2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:16.323998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f2b17-2996-40a5-bf1e-f06adc718503 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:16.325635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 228/454] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Jun 16 00:14:16 2022] GET /identity/v3/groups/e6615e64e1e74047a7ed030299a9c8b2 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:16.332057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:16.427525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb5754-3768-4ffc-a0e8-625f5086fa88 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:16.459612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:16.460465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:16.460755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:16.461066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:16.461893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] RBAC: Adding query filter params (group.id=e6615e64e1e74047a7ed030299a9c8b2, role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=3c21334330db408fb0ba251a2f04ab82) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:16.462207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:16.463619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8436a77-1767-430a-b6bd-0a90c46450d5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:16.469555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 227/455] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Jun 16 00:14:16 2022] GET /identity/v3/role_assignments?group.id=e6615e64e1e74047a7ed030299a9c8b2&scope.project.id=3c21334330db408fb0ba251a2f04ab82&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:19.051182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badc73-f665-467d-8ee9-de8d5e494243 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.051679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badc73-f665-467d-8ee9-de8d5e494243 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.051679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90badc73-f665-467d-8ee9-de8d5e494243 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.053327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 229/456] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:19.059530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce9910-cba3-4090-89cb-e9622c71eee6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.059970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce9910-cba3-4090-89cb-e9622c71eee6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.060169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce9910-cba3-4090-89cb-e9622c71eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.081493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f9ce9910-cba3-4090-89cb-e9622c71eee6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:19.082706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f9ce9910-cba3-4090-89cb-e9622c71eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:19.114124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:19.184687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 228/457] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:19.243067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90badc73-f665-467d-8ee9-de8d5e494243 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.388141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36074f8-2ca5-4f1c-b673-c02b831c551e None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.388670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36074f8-2ca5-4f1c-b673-c02b831c551e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.389123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36074f8-2ca5-4f1c-b673-c02b831c551e None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.394018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 229/458] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:19.395482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90badc73-f665-467d-8ee9-de8d5e494243 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.398426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c0c5-91c0-484d-9bfa-ba95d7a812c1 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.398949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c0c5-91c0-484d-9bfa-ba95d7a812c1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.399382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c0c5-91c0-484d-9bfa-ba95d7a812c1 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.404824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 230/459] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:19.406483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.407015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.407498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.428311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:19.429838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:19.435914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:19.436725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.437037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.437314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.449787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:19.451422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-665dced4-80c8-4855-a820-1b041fd844e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:19.472700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:19.473787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:19.474725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:19.474725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:19.474725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.474725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:19.474725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 230/460] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:19.476299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.476598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.476935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.496264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:19.497805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:19.510090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 231/461] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:19.511222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.556636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:19.605492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:19.606953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 231/462] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:19.607798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.623785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21b0a2-f577-4d81-8413-22acc2979ded None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.672139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:19.672728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.673037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.673314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.674044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:19.674357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:19.675728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:19.688545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 232/463] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:19.689761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.758708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6357d9-c1af-493d-bece-c9014ac6e1ef None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.794301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e59e0-ea4e-4f19-8c9b-cbd8d3f6db80 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.795793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:19.796710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.797105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.797620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.809712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:19.811449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:19.832011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:19.832845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:19.833182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:19.833460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:19.834755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:19.836401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:19.837278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:19.837278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:19.837278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.837278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:19.839429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 232/464] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:19.845350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.846892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:19.848400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:19.856958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:19.870410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:19.870718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:19.871538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:19.872442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:19.872442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:19.872442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:19.872442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:19.876667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 233/465] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:19.877713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:19.963920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db2a58-4571-4d65-b2ac-94317618ac6f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.985639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe96f1-5e15-453d-95a9-ebfa07e2855c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:19.999815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.000495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.000779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.001081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.001787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.003227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.022920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.023703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.024000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.024287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.025301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:20.025637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.037051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 233/466] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:19 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:20.037247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc01c19-6bf2-490d-a86b-e103d8292265 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.037325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.043494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 234/467] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:20.045136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3072f-7d8f-420c-bf04-e4445b16916f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.045547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3072f-7d8f-420c-bf04-e4445b16916f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.045926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3072f-7d8f-420c-bf04-e4445b16916f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.052012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 235/468] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:20.053090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3072f-7d8f-420c-bf04-e4445b16916f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.142514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.146223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.177809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b50176-0b91-4a9b-b2ac-a3fe67befb18 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.210228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3072f-7d8f-420c-bf04-e4445b16916f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.220592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.221378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.221671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.221948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.222690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:20.223008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.224384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6022f5-b7ce-4166-a71e-fea65fa0a5a2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.243928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 234/469] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:20.244994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.245292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.245575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.258451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.259128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.259406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.259680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.260390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.261855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.264467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:20.265720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:20.287459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 236/470] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:20.288945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.352516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 235/471] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:20.380930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.438500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0fce3-62de-4df8-a322-7e0ae794811d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.455490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.487142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.498834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.499410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.499707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.499992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.500865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.502424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbc2e1-2885-4411-897c-0fec8809af03 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.536439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 237/472] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/users => generated 248 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:20.544865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95876f2-a76c-40cb-8aab-710fc7b3f3a7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.572048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d53b7-92c3-4198-a4d1-7108b4e0aec9 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.572379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d53b7-92c3-4198-a4d1-7108b4e0aec9 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.572617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d53b7-92c3-4198-a4d1-7108b4e0aec9 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.574474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 238/473] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:20.579945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.580231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.580512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.600035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.601777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.602257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.602658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:20.602967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.603913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:20.617137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.619975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:20.645900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:20.646984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:20.648517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:20.648517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:20.648517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:20.648517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:20.650803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 236/474] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:20.656379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.697017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 239/475] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:20.725731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.801912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c07ef-c0b9-444c-94e9-5c9ed7e9bd0e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.820558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.851243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1253fc1-b1c9-42b9-8c81-6d7ca96d8e9a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:20.853329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:20.862661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.863581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.863581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.864481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.864664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:20.864979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.866885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.882126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 237/476] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:20.888799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:20.903372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:20.904064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:20.904405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:20.904694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:20.905549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:20.907003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:20.914727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.sql.core [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:14:20.914727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 00:14:20.914727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [parameters: {'id': '6b4c59d350924517bc2631ed748864a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 00:14:20.914727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86549) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result.read() Jun 16 00:14:20.933702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:14:20.934744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application yield resource Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise self.value Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application session.commit() Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:14:20.935828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:14:20.936966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:14:20.937829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result.read() Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:14:20.938714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [parameters: {'id': '6b4c59d350924517bc2631ed748864a0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:20.939668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 00:14:20.940589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:20.946122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 240/477] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:20.950402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:21.087569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c4895-df32-4159-b741-83ea333eb341 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:21.147432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f49c-506c-44c3-b907-eb6191ca4dc9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:21.151142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:21.151822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:21.152106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:21.152382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:21.153134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:21.154585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:21.190368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 238/478] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:20 2022] POST /identity/v3/users => generated 249 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:21.206963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:21.207791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:21.208072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:21.208398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:21.220185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-94deff20-e0ea-488d-b246-bfdc5ad01864 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:21.229505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:21.231627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:21.233186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 241/479] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:21.238688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:21.279420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:21.359279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868773cb-4774-49a4-adb2-a571865c517c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:21.395871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:21.396550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:21.396831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:21.397221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:21.397956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:21.398322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:21.399660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bef624-77ec-4c8c-86d2-e36f338b7b37 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:21.408919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 239/480] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:22.999751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6e03-7565-4b5f-827f-65cf7e729126 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.000250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6e03-7565-4b5f-827f-65cf7e729126 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.000250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6e03-7565-4b5f-827f-65cf7e729126 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.002024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 242/481] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:23.008709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.009271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.009343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.033688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:23.035720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:23.125326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98f8f-cbfd-4a5d-b85c-e5403c3f87fb None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.125621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98f8f-cbfd-4a5d-b85c-e5403c3f87fb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.125911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98f8f-cbfd-4a5d-b85c-e5403c3f87fb None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.128011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 243/482] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:23.133347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d649ea-fe7b-41db-880f-92b40762999a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.133793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d649ea-fe7b-41db-880f-92b40762999a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.134093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d649ea-fe7b-41db-880f-92b40762999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.143371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 240/483] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:23.165357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-62d649ea-fe7b-41db-880f-92b40762999a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:23.167142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-62d649ea-fe7b-41db-880f-92b40762999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:23.169050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.202284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.241703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.306941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 244/484] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:23.308227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46443d-2f5c-4ed3-aa4a-4ee04710e926 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.308523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46443d-2f5c-4ed3-aa4a-4ee04710e926 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.308822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46443d-2f5c-4ed3-aa4a-4ee04710e926 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.310375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 245/485] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:23.315825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8a644-f1a9-488b-baff-b1f768c74fcc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:23.318464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.319757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.320322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.338473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:23.340165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:23.367409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:23.368233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.368526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.368813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.379884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:23.388786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:23.389752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:23.391008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 241/486] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:23.391721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.440969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 246/487] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:23.442245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.526857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.551622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140030f-c684-41a3-a21e-3b124cdefc4b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:23.565920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.589159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175a40c-114a-41b7-8362-3361ae715a73 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:23.607176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:23.608052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.608313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.608596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.629255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:23.630563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:23.632072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.632680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.633190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.634352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:23.634906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:23.636574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:23.638668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:23.639560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 242/488] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:23.645777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.656123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 247/489] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:23.658101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.777832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a8940-0c3b-48ad-ba84-6deb30ecffd2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:23.815187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1cbb-6260-483d-a04a-a9e4ac9fe2f2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:23.818114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:23.819811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.820360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.820846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.833112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:23.842361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:23.843324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:23.844142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:23.852898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 243/490] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:23.854091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.860055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.862871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:23.863561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:23.863931 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:23.864197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:23.865041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:23.865364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:23.867392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:23.886220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 248/491] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:23.886220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:23.901733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:23.985683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d7690-b253-42b7-b10b-76e5334ef47d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.035276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.036184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.036475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.036817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.043803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3881d1-14c9-4e1d-9025-06abcd4e1206 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.049520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.051341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:24.073959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.075045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.076102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:24.076102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:24.076102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.076102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:24.077078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 244/492] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.077619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.092778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.093837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.094318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.095200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.096174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:24.097160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.099098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08299c-fd92-44a1-90bc-27c9264639f5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.114198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 249/493] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:24.117267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3673f2-138a-4417-9897-086dea34bfa6 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.118043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3673f2-138a-4417-9897-086dea34bfa6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.118517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3673f2-138a-4417-9897-086dea34bfa6 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.121192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 250/494] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:24.122163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3673f2-138a-4417-9897-086dea34bfa6 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.154223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.211665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.211856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621345-3447-4b67-a324-620b21a6a874 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.254151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.255504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.256081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.257029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.263199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3673f2-138a-4417-9897-086dea34bfa6 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.272410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.274436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:24.293486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.294509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.295527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:24.295527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:24.295527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.295527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:24.299358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.300253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.302724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.303919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.304199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.305332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.305332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:24.305545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.306901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7494ed-bcca-46f5-af40-c29513760a8b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.324295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:24.324295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.324295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.324295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.337719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:24.338921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:24.401408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc684a6b-f799-442c-85c0-cf21e1721e32 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.403398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 252/497] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:24.404224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.424827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.450292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.451100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.451391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.451673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.462141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.462854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.463635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-03e3e841-8f9f-493e-bff7-a584777ba63f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:24.482389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.483453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.484475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:24.484475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:24.484475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:24.484475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:24.484475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 246/498] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.484801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b23f4-3616-4992-834a-75184b0ca051 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.485142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b23f4-3616-4992-834a-75184b0ca051 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.485428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b23f4-3616-4992-834a-75184b0ca051 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.487632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 247/499] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:24.487632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b23f4-3616-4992-834a-75184b0ca051 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.507955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f7f8-7811-43f5-96a3-cb6155d5e3ac None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.543862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.544537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.544827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.545138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.545873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:24.546194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.547569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.559872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 253/500] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:24.560673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.591651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b23f4-3616-4992-834a-75184b0ca051 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.626083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.626891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.627182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.627463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.635813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.637311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:24.653568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.654557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:24.655427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 248/501] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.655989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.665653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc98502-af18-4dbc-a6a2-e0d9d022dc93 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.680143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.685510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.701643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.702440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.702725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.703001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.718367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0ee55634-e7b7-42dc-b2ca-2ed556902ab2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:24.726207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.727186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:24.727968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 254/502] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.728603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.728955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.729276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.746205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:24.747423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:24.764762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbc86a-3470-4174-9d43-631136bf451b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.797674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.798349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.798638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.798915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.799647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:24.799957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.801374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.812981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 255/503] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:24.813626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.818432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 249/504] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:24.818432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.899735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.914121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34712e-b534-46de-9aac-dac2b3a7d5b7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.919014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffdb3-681d-49a1-9abb-ca73ef6d2a32 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:24.929798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:24.953133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.953924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.954211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.954486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.954748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:24.955423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:24.955713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:24.955996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:24.956843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:24.957203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.958628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.961588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:24.963026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:24.979097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:24.980239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 256/505] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:24.986225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:24.995308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 250/506] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:24.996146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.087743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a852e-9c55-4068-b8a7-c5c49224dda3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.098136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8d0dc-e8de-42c2-aff8-efd5a7833c83 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.122241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.122997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.123285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.123572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.124327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:25.124641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.126062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.132640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.133469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.133760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.134041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.134731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 257/507] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:25.135523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.141657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.143132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.144914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:25.159392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:25.160244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:25.161181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 251/508] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:25.161724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.169467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.237578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193313c-dc42-49c2-afdc-fb83a2636202 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.258556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85d1f7-7d22-4c54-9310-0af55153190b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.270307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.271111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.271402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.271687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.282560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.284334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.291984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.292644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.292958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.293242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.294072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:25.294384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.295805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:25.302237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:25.303038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:25.304117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:25.304117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:25.304117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:25.304117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:25.304117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 258/509] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:25.304371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.320488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 252/510] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:25.320488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.371501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.389116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.404826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3524c-b32e-4690-8813-b7ba947b5b30 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.416020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042db1e1-ddc9-4106-bbc3-9736ba8dfe10 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.438148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.438796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.439084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.439358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.450669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.451466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.451755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.452031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.462714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.464165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:25.481516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:25.482356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:25.483195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:25.483195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:25.483195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:25.483195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:25.483195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 253/511] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:25.483468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.511456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.513024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.582524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3c9c3-6681-4185-b4e2-a582c58070c1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.592063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.597364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.616672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 259/512] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:25.617527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.623619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.624307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.624596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.624897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.626170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:25.626491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.628625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.645960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 254/513] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:25.645960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.724108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e33bd7-7b72-4660-bbfb-1c1023343a56 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.748302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516890e-923d-440a-8c8e-8dfd0ee10b66 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.759319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.760183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.760463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.760741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.761491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:25.761805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.763188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.774411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 260/514] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:25.775457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.783478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.784130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.784412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.784688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.817768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.822889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:25.851900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.853406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.878145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f2ac-2894-4257-bede-59d2dbb7b46f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:25.910526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:25.911180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:25.911466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:25.911747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:25.912466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:25.913472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:25.915027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:25.926527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 261/515] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:25.927324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:25.961532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 255/516] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:25.962181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.027433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.034657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd46c2-413c-4bb9-97c0-8f50eb539237 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.037398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.063739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ac0a6-2e41-4311-a284-2c38eea576d2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.069712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.070588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.070874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.071151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.081441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.082901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.099963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.100635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.100954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.101239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:26.101874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:26.103055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:26.103874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:26.103874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:26.103874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.103874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:26.103874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 262/517] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:26.104151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.170222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.171700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.202193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6c5e6-ca37-4f5f-815a-c6c642c6570c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.232712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.234547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.235268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.235556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.235832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.236528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.238000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.257462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 263/518] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:26 2022] POST /identity/v3/users => generated 247 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:26.258266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.276493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.281206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 256/519] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:26.281796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.387856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f896d-d937-4b27-a8db-24b3248248dd None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.404063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8044bb8-8f4b-44ef-938a-579bfa08fd30 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.425705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.426195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.426479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.426751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.435019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.436462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.448589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.449449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.449733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.450012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:26.453493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:26.454348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:26.455256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 264/520] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:26.461409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.465218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.465218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:26.486012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:26.486880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:26.487705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:26.487705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:26.487705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.487705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:26.487705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 257/521] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:26.488022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.508287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.543299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.576753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11f90b-f683-4567-ba2d-02389b6704e6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.597397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a383aa-4648-4dc7-b59f-c783bf1cd96c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.612250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.612947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.613240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.613518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.614247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:26.614560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.616479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.628654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 265/522] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:26.633821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.636899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.637709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.637991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.638267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.650704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.652254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:26.671663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:26.672844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:26.673893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:26.673893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:26.673893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:26.673893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:26.675908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 258/523] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:26.676829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.752741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df384d-1edc-40ef-888f-c90f65a22184 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.760826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.787119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:26.794106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.794722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.795008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.795344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.796188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:26.796529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.798018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.798711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e9532-3029-43e7-a229-232df8288404 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.840103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 266/524] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:26.840103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.841578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.842145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.842425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.842700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.843432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:26.843743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.845158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.858094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 259/525] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:26.859045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:26.949332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7c2bf-74eb-444b-a296-49b4db36656f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.967882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986db883-e448-480d-9a9b-f1aebad08cb3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:26.985977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:26.986783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:26.987070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:26.987347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:26.994609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:26.996041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:26.999551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.007523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.008265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.008561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.008844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.009608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.009923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:27.011077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:27.011899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.017739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:27.018186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.018268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 267/526] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:27.018268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.037412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 260/527] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:27.037412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.065373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.120936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445a258-2cbd-4924-97d7-a503596a7f2e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.156196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ab6e-2051-4add-baf7-6fdaede8f331 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.157916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.158568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.158858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.159138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.193098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.194417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.194910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.195384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.202419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.204075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:27.218965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:27.219940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.220995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:27.224911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 261/528] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:27.225824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.230467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.231935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.282346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.286183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.336934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 268/529] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:27.337538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.356949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde1b47-21a5-4668-aca0-3f8e7112a718 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.392926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.393596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.393880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.394160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.394952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.395266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.396688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.420929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 262/530] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:27.420929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.454353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef863e-c07e-4c87-bdbd-9e7b718e758b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.493372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.494275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.494565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.494841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.502025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.503462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:27.517606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:27.518456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:27.519278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 269/531] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:27.524089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.530953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac193b34-6c1a-4bda-8395-0b5a313ae244 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.538227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.545214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.571885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.572554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.572836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.573155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.574137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.574324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.575700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.588100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 263/532] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:27.588834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.641720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabbf85-089b-4ab8-aae5-21c59ab1b53c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.678877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.679786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.680215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.680756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.695011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.696446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:27.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:27.718288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:27.719190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:27.719190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:27.719190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:27.719190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:27.719190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 270/533] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:27.719697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.724378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafed2e-9a35-4ce9-848d-654cff7c90d5 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.765701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.766753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.767037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.767312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.768258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.768573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.770045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.776559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 264/534] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:27.785946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.787143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.836544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1365a-7d82-46b0-8e67-bbc822bd7365 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.845645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:27.879316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.879998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.880288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.880570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.881653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.882490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.884234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:27.906173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 271/535] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:27.907191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:27.944986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f8b2c-6cff-432f-b8d2-14b1f720ab49 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:27.994747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:27.995414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:27.995707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:27.995990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:27.996911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=8863fd88c1a2416c9fdb0148db03f6e5) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:27.997194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:27.998788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.004967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 265/536] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/role_assignments?user.id=8863fd88c1a2416c9fdb0148db03f6e5&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:28.005675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:28.042517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caaa58d-2d71-49f1-93d1-dce44fabf2c7 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.073691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:28.117837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.118668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.119177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.119656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.128909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:28.128909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:28.128909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.153942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:28.160818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4708b2-e420-47b3-9ec5-95d012de6c28 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.164189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:28.172420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:28.213384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.214062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.214350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.214634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.219567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=ea2357b286df4e828b9f5024d71544ab) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:28.220945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:28.222990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.229713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/role_assignments?user.id=ea2357b286df4e828b9f5024d71544ab&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:28.321957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b61959-a7a7-4dc9-a427-ee7d830ea2be None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.359706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.360900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.361408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.361875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.369866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:28.371706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-be5f5522-f7d8-4561-9d4f-982d0f11ef4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:28.387603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:28.388489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:28.389371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:28.391647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 273/539] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:28.398059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:28.526272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0861-047b-4ce9-b603-deae29029607 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.526956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0861-047b-4ce9-b603-deae29029607 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.527436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d0861-047b-4ce9-b603-deae29029607 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.529274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 274/540] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:28.533882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c2f2-3e2e-4c80-b734-7e93317ca656 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.535361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.535849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.536339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.555776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:28.557422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:28.571833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.572519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.572810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.573124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.573933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:28.574255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:28.575791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.589916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 267/541] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:28.595668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:28.609295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:28.640625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 275/542] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:28.645589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:28.668340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:28.776677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc85e1-8986-4e7a-8270-2f662be9846f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.812834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.813667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.814049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.814445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.816990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e1d08f8e3b4d4376aff855f4bce68c8c) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:28.816990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:28.817303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:28.823794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 268/543] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/role_assignments?user.id=e1d08f8e3b4d4376aff855f4bce68c8c&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:28.845926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849663d7-fedd-44a3-a24d-7a3f88641281 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:28.884653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:28.885757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:28.886225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:28.886661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:28.898355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:28.906783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:28.907886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:28.908909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:28.911294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 276/544] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:28.917742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:29.041258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:29.049221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa38fd7-0205-4513-8c07-46548cd34f09 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:29.091884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:29.092813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:29.093265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:29.094177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:29.095095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:29.095582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:29.097172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:29.112245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:29.113341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 269/545] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:29.157066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:29.261189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430205e3-4726-4ee2-acc1-beccbbb92905 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:29.295943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:29.296752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:29.297065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:29.297343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:29.309539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:29.311230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:29.329592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:29.330652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:29.331641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:29.331641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:29.331641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:29.331641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:29.331641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 277/546] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:29.337580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:29.462887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77263-3c36-4ecd-a191-2fc8d0861342 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:29.499436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:29.500467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:29.501031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:29.501511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:29.502437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:29.502948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:29.504529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:29.518055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 270/547] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:29.524383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:29.650610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:29.671988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98678dc-d53c-461a-bf42-4cbb36036ccc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:29.742195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:29.743026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:29.743318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:29.743596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:29.755138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:29.757179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:29.771503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:29.772451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:29.773877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 278/548] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:29.780939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:29.797237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:29.905322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b3e37-65a9-4fbe-b0a0-2063853575fa None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:29.940221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:29.940922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:29.941223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:29.941505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:29.942348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:29.942660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:29.944668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203ac87-c806-45cd-9b03-765e27bd5dec None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:29.962491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 271/549] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:14:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:29.968030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:30.132940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71f35-5b15-4dad-8e0d-9792d33f2f2c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:30.180361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:30.181174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.181428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.181713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.197268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:30.247802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020131-b746-4fba-bde8-7fb8579269e3 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.248108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020131-b746-4fba-bde8-7fb8579269e3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.248368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020131-b746-4fba-bde8-7fb8579269e3 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.250004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 272/550] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:30.253200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:30.254657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:30.258153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.258443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.258726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.305908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:30.307128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:30.391376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 273/551] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:30.392427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:30.395003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 279/552] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:30.400974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:30.444245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:30.521244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:30.599448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614da69b-3e71-4182-8a8b-e6ba39e7edcf None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:30.623350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1749e-4b67-4099-9221-250a6f0f2588 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:30.669690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:30.670510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.670799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.671170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.684079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:30.685445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.685945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.686425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.688987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:30.689133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:30.709011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:30.728261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:30.729284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:30.730129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:30.730129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:30.730129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:30.730129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:30.732898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 274/553] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3bdf2182-912d-4784-9ff3-3a6cbbc9efb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:30.739075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:30.740134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:30.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 280/554] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:30.743787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:30.744604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019cc6b-da2f-472c-b6bd-f9d84a547918 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.745027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019cc6b-da2f-472c-b6bd-f9d84a547918 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.745363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019cc6b-da2f-472c-b6bd-f9d84a547918 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.747051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 281/555] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:30.751338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019cc6b-da2f-472c-b6bd-f9d84a547918 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:30.809197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:30.865359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2640-321b-46fd-830b-2dd749617459 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:30.891814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019cc6b-da2f-472c-b6bd-f9d84a547918 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:30.896115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:30.906905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:30.907724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.908088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.908442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.909358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:30.909742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:30.911219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc3081-fc19-409b-a700-bc7a0572c591 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:30.931227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 275/556] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:30.932637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea421-b802-4486-a20b-db5dbc0427b8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.933066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea421-b802-4486-a20b-db5dbc0427b8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.933353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea421-b802-4486-a20b-db5dbc0427b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.945315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:30.946278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:30.947162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:30.947446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:30.948268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:30.948584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:30.950114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:30.953429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-755ea421-b802-4486-a20b-db5dbc0427b8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:30.954641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-755ea421-b802-4486-a20b-db5dbc0427b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:30.959016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 282/557] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:30.961070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.032064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 276/558] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:31.038300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced94f-a707-4d1f-b995-5d9ddd5ba207 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.038598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced94f-a707-4d1f-b995-5d9ddd5ba207 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.038882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adced94f-a707-4d1f-b995-5d9ddd5ba207 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.040599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 277/559] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:31.040867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adced94f-a707-4d1f-b995-5d9ddd5ba207 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.080308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c98f5e-282d-4abd-97fd-a1253a73f6a0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.089935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.118787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.120065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.120539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.121161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.133668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.135128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-ff6c74dc-f158-48f2-baeb-3f69b400ef79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:31.151010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:31.151884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:31.152736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 283/560] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:31.153862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.153862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.154167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.169052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adced94f-a707-4d1f-b995-5d9ddd5ba207 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.171819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:31.173021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:31.194716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.206203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.207007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.207298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.207583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.219521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.221041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:31.241295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:31.242308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 278/561] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:31.248081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.267136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 284/562] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:31.277077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.363492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.374438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276684f-0c33-4ed7-8b5f-af1811f2167f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.386420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386d317-2189-4367-9ed7-c40f67afa193 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.411741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.413276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.413613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.413900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.423039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.423931 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.424383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.424613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:31.425068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.426026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:31.426509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.428688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:31.433228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:31.434225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:31.435047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 279/563] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:31.435718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.447153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 285/564] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:31.448019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.471026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.547629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96b476-1f96-4279-a847-0dd355cfb07e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.560027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7445590-8340-43f4-9c91-d23bc890557d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.586714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.588323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.588606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.589511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:31.589824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.591237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.596541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.597398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.597683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.597961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.607233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 280/565] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:31.607924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.618256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:31.622694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:31.626269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:31.627199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:31.628005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 286/566] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:31.628684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.718629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.734341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df333ba-f0a5-42b3-be2e-d7f180c9ffda None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.742492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0805dfd-0503-44f7-a7d3-d77965fd2459 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.768614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.769318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.769607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.769891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.770654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:31.770973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.773131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.776350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.777062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.777344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.777627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.778463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e57e080ca56d4f28bb1692d147cf4caa) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:31.778782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.780197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.781232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 281/567] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:31.782019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.786384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 287/568] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/role_assignments?user.id=e57e080ca56d4f28bb1692d147cf4caa&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:31.787079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.864378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:31.899783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f00454-fe10-459c-b561-3d441c3dfb5c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.910198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa28cc-0e3b-49f6-a864-9712da642925 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:31.937821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.938996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.939289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.939574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.949566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.951126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.951322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:31.952119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.952507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.953045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.954474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:31.955041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:31.957689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1208342-992c-4cf6-9261-40131b85a46c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:31.973946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 288/569] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:31.974116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:31.975046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:31.979974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:31.980827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76669f-bf20-4689-bac9-0dd4b73a0a12 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:31.980827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76669f-bf20-4689-bac9-0dd4b73a0a12 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:31.980827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76669f-bf20-4689-bac9-0dd4b73a0a12 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:31.980827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 289/570] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:31.980827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76669f-bf20-4689-bac9-0dd4b73a0a12 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.981604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 282/571] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:31.981604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:31.986497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.094680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a423dc-d278-44e1-bb9f-5d04a40e269c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.096595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76669f-bf20-4689-bac9-0dd4b73a0a12 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.126548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.133302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.134134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.134263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.134486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.134937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.135078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.135209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.135510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.135857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:32.136224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.138184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1572c37-31ab-46fa-9a18-767950c9f973 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.148946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.150651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.157347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 283/572] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:32.158686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.158988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.159274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:32.173698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:32.174612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:32.175578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:32.175578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:32.175578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.175578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:32.176873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:32.177340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 290/573] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:32.178296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:32.178703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.248305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.255202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 284/574] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:32.262375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.299176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88809d-2635-4291-be67-8c92ae746153 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.339371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.340177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.340468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.340747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.352297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.354107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:32.372398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:32.373319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:32.374220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:32.374220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:32.374220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.374220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:32.374220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 291/575] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:32.374522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.390236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf089b-6fe5-47ed-9f7b-2de4f7e44f6e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.412349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.427254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.428547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.429266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.429266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.440384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.442084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:32.462848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:32.463767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:32.464640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:32.464640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:32.464640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:32.464640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:32.464640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 285/576] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:32.465432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.486816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127c0af-0933-4606-9405-6a11670b7268 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.494651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.524297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.524905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.525463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.525758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.526480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:32.526795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.528173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.544764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 292/577] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:32.544764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.575862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4234f6f-1187-4279-a695-ef2f5c221129 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.610790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.611898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.611898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.612157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.624499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:32.633187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:32.634279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:32.635325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:32.639213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 286/578] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:32.640035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.667160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513fbcb-ec67-4c69-aec4-3f1f8eefcc17 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.672628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.707950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.708624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.708932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.710036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.710843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:32.711161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.712569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.730311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 293/579] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:32.731376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.753177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc73cc-2f09-405e-888d-8a6b6661a52d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.761461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.793462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.794320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.795022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.795320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.796345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:32.796684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.798985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.811187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 287/580] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:32.812297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.844731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1da3ee-3b31-4b39-ae57-79d0096a53e5 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.884044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.885333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.885934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.886218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.894612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.896110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.907059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:32.912902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:32.913898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:32.914820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 294/581] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:32.915467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:32.920567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21883ee-2184-4d7a-88af-984414bc1dde None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:32.959160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:32.959864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:32.960345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:32.960626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:32.961413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:32.961723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:32.963129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:32.971887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 288/582] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:32.972703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.015632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.033649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf291-2878-41be-9187-9daaba91cae7 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.074327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.075125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.075409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.075680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.076528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91fa5b-446a-4dce-a6d7-b059bd56811f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.083576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.085289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:33.099548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:33.100721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 295/583] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:33.101947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.120848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.122433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.122922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.123398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.133151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.135500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:33.153518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:33.154381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:33.155242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:33.156436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 289/584] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:33.156615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.172425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.217877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39cc91-cad4-49d1-b784-a83171a510fc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.255482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.256453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.256818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.257190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.258107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:33.258497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.260071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.264041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb6a1c-cf1f-41e6-aff5-043f20db66f2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.279632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 296/585] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:33.280407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.285444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.306400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.307199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.307480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.307756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.319802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.320662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:33.338785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:33.339726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:33.340635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:33.340635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:33.340635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.340635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:33.340635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 290/586] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:33.341608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.392986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70054b3-759a-479c-95b9-1e140ef1fd18 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.410705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.429637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.430320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.430610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.430894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.431744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:33.432057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.433528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.449936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889a8b5-172b-4c35-aed8-99c8926b57e2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.451448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 297/587] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:33.452210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.492385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.493084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.493371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.493648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.494440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:33.494752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.496192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.513020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 291/588] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:33.513020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.527377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.570264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc1a0c-f762-4c28-b336-edd7b6b46fea None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.606643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.607363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.607647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.607919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.618746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.620184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.622302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c364f4-081e-477a-ae88-7578cf90732a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:33.638757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:33.639684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:33.640537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:33.640537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:33.640537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.640537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:33.640537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 298/589] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:33.640938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.657799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.667187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.667864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.668146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.668423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.669182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:33.669492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.670850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.683075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 292/590] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:33.683850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.767178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6931b8-aca9-49b2-ab0c-33ed3cfa7e9f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.783072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.794025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ff935-08b8-4828-8aab-bff888bc8dd6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:33.804140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.804974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.805264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.805541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.816092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.817563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.832437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:33.833790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:33.834788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:33.835264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:33.835798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:33.836648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:33.837703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:33.837703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:33.837703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:33.837703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:33.837703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 299/591] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:33.838289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.860850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=d50db388db1446eeb60ccaa17ff55389, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:33.862266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:33.877291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 293/592] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:33 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:33.878483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:33.928629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:33.981616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5cba3-bf3a-4aca-a3e2-8d0857428165 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.007096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f865d76-7433-408c-a59e-f8f4c0cadc17 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.019957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.020645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.021134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.021424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.023418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:34.023741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.025442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.043010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 300/593] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:34.043010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.057739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.058566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.058903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.059185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.066929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.068374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.075398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-425538a9-04ef-45e1-a419-56b18eb89b3d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:34.083793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:34.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:34.086020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 294/594] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:34.086696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d38c3-5997-469d-98fc-36dec845a999 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.086849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d38c3-5997-469d-98fc-36dec845a999 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.087125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d38c3-5997-469d-98fc-36dec845a999 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.088588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 295/595] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:34.093223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d38c3-5997-469d-98fc-36dec845a999 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.163743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb9b0-8bde-4c8a-8080-a7acaa2f5144 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.189233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.196990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d38c3-5997-469d-98fc-36dec845a999 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.201290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.202157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.202608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.203035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.203927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:34.204400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.205801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.220760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 301/596] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:34.225460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.237125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.237939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.238227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.238509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.246144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.247664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-da80ef6d-099d-44df-a037-506bec318a37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:34.262358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:34.263299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 296/597] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.269517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.270143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.283873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:34.285088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:34.319924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.335054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762593e-1508-48f9-8209-825598cb2e57 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.364929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 297/598] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:34.366722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.384961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.385652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.385939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.386212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.387058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:34.387373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.388802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.411192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 302/599] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:34.411192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.479634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.497153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2b631-cc12-43cd-b675-1b9162bed29a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.514116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aed6bb-f6ee-491a-a422-bbc1a0c6f685 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.533408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.534206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.534489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.534761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.542348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.544444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.552712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.554542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.554829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.555108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.555903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:34.556215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.557664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:34.559069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:34.560079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 298/600] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:34.566125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.585632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 303/601] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:34.585632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.587714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.672236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dc811-460e-42fd-b464-e0387ca41f13 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.699581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23055044-2a81-46bd-ae4f-05f7f0354840 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.718811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.719696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.720042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.720385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.731465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:34.739161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:34.740267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:34.742124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:34.743387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 299/602] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:34.744162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.745169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.745520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.745875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.749089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.759022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.759365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.760604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:34.780676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:34.781753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 304/603] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:34.789163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.846190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:34.862114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b20d23-c708-4ff5-8249-496264157fdc None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.893289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e8028-e8cd-47cb-a8d7-00c899b43b4a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:34.895927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.896793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.897215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.897586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.898469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:34.898881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.900648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.918050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 300/604] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:34.918050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:34.934548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:34.935531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:34.935904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:34.936297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/8863fd88c1a2416c9fdb0148db03f6e5/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:34.951886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=8863fd88c1a2416c9fdb0148db03f6e5, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:34.953452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:34.966981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 305/605] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:34 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/8863fd88c1a2416c9fdb0148db03f6e5/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:34.967860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.009216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.076873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2041-4508-432c-bad3-b9b9d34d0d52 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.149226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183e725-270c-4d01-9b4f-8caa77e82a18 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.151331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.153187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.153687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.154434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.155557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:35.156058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.157658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.158567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.173536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 301/606] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:35.174526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.202043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.202721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.203053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.203338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.204068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:35.204378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.206546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.222478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 306/607] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:35.223157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.329163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519032b-a152-4cc3-b45d-9dc25c9342fb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.369283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.379167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.380195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.380585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.380863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/ea2357b286df4e828b9f5024d71544ab/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.397859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f62d61-a6c5-4ecb-87c4-ab2f8527bf11 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.414740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=ea2357b286df4e828b9f5024d71544ab, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.419512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.437021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.438819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.439345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.440278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.447042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 302/608] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:35 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/ea2357b286df4e828b9f5024d71544ab/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:35.449120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.472797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.474407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:35.533907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:35.534895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:35.535838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:35.535838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:35.535838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:35.535838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:35.535838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 307/609] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:35.541704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.645105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.707708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aef1b-716d-4df8-a951-22baea2192e8 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.751375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.757686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.757985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.758271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.766231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.767687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.768517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed9ec-ab54-4d7c-a2d0-e7777273e208 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:35.793493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:35.794405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:35.795255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 303/610] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:35.808644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.810546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.811277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.811687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.811972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.812703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:35.813142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.814520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.828409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 308/611] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:35.835381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:35.861673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.926768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831d1e-580b-48c8-abde-cfa9554a4711 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.944057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:35.965361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:35.966119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:35.966407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:35.966709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:35.967688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:35.968006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:35.969495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:35.981042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d990cbe-7000-4988-b387-aab90db252a7 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:35.982834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 304/612] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:35.995165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.024565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.025705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.025988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.026261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.034606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:36.036147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:36.052379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:36.055096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:36.057263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 309/613] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:36.064908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.110845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c010-0965-4834-9017-73c84d33c8fd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.113310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:36.165881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.166810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.167108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.172901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e1d08f8e3b4d4376aff855f4bce68c8c/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.184840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=e1d08f8e3b4d4376aff855f4bce68c8c, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:36.186374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cf3e2-00e6-42d7-990f-bce9a1ab1450 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:36.206415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 305/614] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:35 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e1d08f8e3b4d4376aff855f4bce68c8c/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:36.242477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3efe24-28b4-47b8-97ee-1da5b8f79e96 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.245919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643388-df0c-4146-a01c-9cb06d05d547 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.246219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643388-df0c-4146-a01c-9cb06d05d547 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.246503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae643388-df0c-4146-a01c-9cb06d05d547 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.248189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 306/615] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:36.254618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.255051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.255434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.274528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:36.276168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:36.296275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.297281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.297773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.298243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.299257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:36.299725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:36.301494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:36.311941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:36.327188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 310/616] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:36.332667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.401799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 307/617] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:36.426376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.526948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c717f0-eb5a-4982-badc-89349fda16a2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.551340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:36.589797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.590616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.590906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.591187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.593924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfe774-b38e-4018-9936-46bc085c9e88 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.613206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:36.614669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:36.648612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:36.649576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:36.650721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:36.650721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:36.650721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:36.650721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:36.657485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.658529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.659084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.659562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.661811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 311/618] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:36.663692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.681673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:36.691837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:36.692790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:36.694323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:36.699616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 308/619] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:36.718938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.769716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:36.812605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1effa99-7a5e-4c5e-94db-fe11b279f8bc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.865744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.867078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.868637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.869316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.870850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:36.871567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:36.873167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:36.890984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 312/620] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:36.895979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:36.939482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629af8a-471c-4246-a9b3-546996ceb46a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:36.995205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:36.996274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:36.996759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:36.997258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:36.998281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:36.998786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.000387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.006569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.013027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 309/621] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:37.018922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.116002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ccc01-9d4b-408f-af02-ce52393047cb None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.137241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.190901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524041d-43af-4773-80cf-e1dbe5ed757c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.193537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.194353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.194635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.194913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.207402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.208849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:37.236597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:37.237583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:37.238451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:37.239892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 313/622] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:37.247925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.265067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.265882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.266165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.266442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.279186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.280659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:37.316749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:37.317714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:37.318696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:37.318696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:37.318696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:37.318696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:37.318696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 310/623] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:37.329810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.379474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6ef2e-1e52-45c1-a259-961e6f54bd19 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.421701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.422373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.422662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.422941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.423740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:37.424054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.425533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.456316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 314/624] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:37.458604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.465238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.519294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.532382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257094d-3728-4c6a-9f19-1e0b6c4e2ce1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.584991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.585763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.586049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.586329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.587066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:37.587380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.588757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.603238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d1e72-e661-454d-bd5f-85b5d9e709e2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.613638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 311/625] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:37.619562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.644109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.645387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.645874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.646352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e57e080ca56d4f28bb1692d147cf4caa/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.672339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=e57e080ca56d4f28bb1692d147cf4caa, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.674483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9cd93-52ff-4001-bd86-bc72b8b6e2e6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.696993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 315/626] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:37 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e57e080ca56d4f28bb1692d147cf4caa/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:37.699125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924fe96-43c7-46f0-b1e6-4d0f4a2ffd99 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.699667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924fe96-43c7-46f0-b1e6-4d0f4a2ffd99 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.700196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924fe96-43c7-46f0-b1e6-4d0f4a2ffd99 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.703320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 316/627] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:37.715949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.716575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.717095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.741132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:37.743097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:37.852203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883424d9-4081-4fd2-a4ea-a7ca2f271421 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:37.879917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 317/628] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:37.881361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.909710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:37.920502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:37.932747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:37.933715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:37.936855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:37.937266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:37.950049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:37.951954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:37.973187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:37.974399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:37.976491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:37.980984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 312/629] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:37.987536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.062079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afbb14-8439-42ea-a033-ba75b7b55c61 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.098817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.100211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.101056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.101367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.112091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-2ddfe28a-7c93-41a1-9ffa-db7afcbd938e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:38.121079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:38.122014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:38.122990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 318/630] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:38.123829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f76e22-468c-46e1-8f45-58cfee7665ab None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.124124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f76e22-468c-46e1-8f45-58cfee7665ab None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.124405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f76e22-468c-46e1-8f45-58cfee7665ab None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.125295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063af27-a869-412e-bb5f-4be3641faace None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.125998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 319/631] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:38.129715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f76e22-468c-46e1-8f45-58cfee7665ab None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.163986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.164928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.165387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.165828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.166839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:38.167317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.169053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8fa78-8efc-46df-a482-ab90d85d9157 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.190273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 313/632] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:38.191922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.192402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.192926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.209209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.211230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:38.212723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:38.239661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.247842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f76e22-468c-46e1-8f45-58cfee7665ab None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.287527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.288653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.288964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.289246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.290002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:38.290516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.291681 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.295692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 314/633] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:38.296784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.308997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 320/634] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:38.314137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.415527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279ff44-7766-4243-9e71-c5d51cd76022 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.451764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3eab4-927f-4096-af8f-102baf8364b6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.465950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.467303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.468192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.468559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.483629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.494672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.496632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.497117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.497560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.509647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.510485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.511524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:38.533246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:38.534172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:38.535041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:38.535041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:38.535041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:38.535041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:38.537677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 321/635] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:38.538816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.561357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.562834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.653549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16c6b5-1cf1-4430-8782-5b662422a331 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.695642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.696780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.697422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.697922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.707974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 315/636] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:38.710421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.717427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a3d6f549-0fdd-43fa-b549-34743987efff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:38.725691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:38.726624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:38.727544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 322/637] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:38.728842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71461-ddbb-41ae-942c-9b30938a337d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.728842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71461-ddbb-41ae-942c-9b30938a337d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.728842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71461-ddbb-41ae-942c-9b30938a337d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.733150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 323/638] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:38.733150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a71461-ddbb-41ae-942c-9b30938a337d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.779517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.784397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:38.829517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9ce34-83d4-4eda-9d88-94ea78536e7d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.842677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a71461-ddbb-41ae-942c-9b30938a337d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:38.867784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.868476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.868782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.869059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.869784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:38.870239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.871469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.877444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:38.878249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.878532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.878811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.888372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 316/639] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:38.888372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:38.890394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:38.891941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-7df1f357-eb37-4b5a-bc8a-abed9cce3ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:38.909615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:38.910568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:38.911420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:38.911420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:38.911420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:38.911420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:38.916272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 324/640] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:38.917567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:38.917857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:38.918142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:38.934337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:38.935507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:38.999849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24510424-4d7f-4685-bdaf-d86aad6b8284 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.008944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 325/641] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:39.008944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.037358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.037554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.052475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.053699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.053987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.054265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.055692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.056016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.057431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.070884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 317/642] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.070884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.123984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c9f2-11c8-40d8-a708-be545e41dd86 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.161890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.163004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.163450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.163732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.169570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6534f-a777-4292-b18c-b7e27bb2d469 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.171133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.172582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:39.187819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:39.188847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 326/643] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:39.195281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.211941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.212673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.213057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.213411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.214207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.214591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.215962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.228602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 318/644] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.229247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.276117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.284658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.301054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2392230-99be-453a-81e6-09bc09bc3f25 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.334974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.335768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.336054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.336331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.337417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9cdc0-5a53-431a-802e-6a56f88c0fce None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.346301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:39.354663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:39.355770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:39.356710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 327/645] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:39.357576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.379483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.380282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.380576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.381842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.392333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.393824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:39.412895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:39.413829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:39.414741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:39.414741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:39.414741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:39.414741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:39.414741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 319/646] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:39.415497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.466345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa6038-cced-48c5-8c05-8ab3938a0c9b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.501496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.502195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.502479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.502755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.503589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.503905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.505384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.510989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.519042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e71d5-cc25-4bf7-a564-d27b9e27fa3e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.529723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 328/647] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.529723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.529723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.560972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.561776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.562167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.562449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.570103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.571707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:39.586413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:39.587316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:39.588200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 320/648] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:39.588671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.638805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c55b80-b924-479f-81d7-b7e22bb7880c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.676123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.677549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.678002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.678290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.679042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.679353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.680732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.693922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 329/649] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.693922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.700295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9ff64-56b0-469a-bfc1-a3232610bebb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.737204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.737868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.738144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.738426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.739149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.739456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.741716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.758451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 321/650] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.759322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.760246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.776221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:39.805637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e850-6727-45fd-8be3-3e8a1c9a906b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.841961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.842619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.842912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.843197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.866273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537237e1-3038-4ddd-8be4-46a91cd3ec0c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:39.906505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:39.907814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:39.908112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:39.908396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:39.909869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:39.910203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.911918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.916399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:39.917982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:39.930055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 322/651] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:39.930055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:39.990980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.016014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.033997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 330/652] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:40.037859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.043603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476067f-d887-401a-ba3c-960b31231ef0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.077773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.078565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.078853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.079135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.090001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.091442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:40.108546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:40.109468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:40.110351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:40.110351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:40.110351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:40.110351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:40.110351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 323/653] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:40.110632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.136358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24751d-ee33-4fd3-bb71-f40729243356 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.170311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.171111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.171398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.171678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.179569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.181039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-30964e38-d9db-4ccb-a152-59967a5d1f01 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:40.195499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:40.196602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:40.197773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 331/654] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:40.198347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384290-efe6-49aa-8f26-fdbad4eb74d2 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.198347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384290-efe6-49aa-8f26-fdbad4eb74d2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.198464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f384290-efe6-49aa-8f26-fdbad4eb74d2 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.199833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 332/655] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:40.200529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f384290-efe6-49aa-8f26-fdbad4eb74d2 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.219104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d356ce-c981-4212-bac0-50eb12eeaaef None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.231186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.244591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.253414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.254087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.254373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.254656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.255502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:40.256371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.257843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.263775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 324/656] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:40.264556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.321451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f384290-efe6-49aa-8f26-fdbad4eb74d2 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.360639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.361398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.361742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.362019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.373244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.375029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.394519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69a76-f3ec-4a59-922f-0c648012be45 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:40.394892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:40.395750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:40.397135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:40.397135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:40.397135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:40.397135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:40.397135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 333/657] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:40.397434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.441308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.441987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.442276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.442555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.443614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:40.443733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.445156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590205b0-6b5f-4873-8902-464bba02ad2d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.463446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 325/658] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:40.463446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.463446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.463446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.480322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:40.481572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:40.500021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.526130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1edc-2503-40e2-a5d2-d2ec89eaad7c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.536154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.560277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 326/659] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:40.561237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.567779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.568981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.569505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.570041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.571658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:40.572185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.574321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.595446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 334/660] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:40.596756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.678537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879256-ce66-434f-bea6-3b79196e5c91 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.705081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29aabe-17fc-403c-9302-4386c820f487 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.716828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.717524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.717820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.718090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.718820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:40.719135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.720530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.735273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 327/661] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:40.736111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.745338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.746154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.746448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.746734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.748845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.754597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:40.757161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:40.772799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:40.773754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:40.774664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 335/662] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:40.775511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.788300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.855263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1976b-5514-4cbc-a823-6f98c28c775b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.884657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421b81b-6349-419f-8884-ed9b5d418b72 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:40.893148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.893959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.894517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.894517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.907219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:40.915275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:40.916170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:40.917154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 328/663] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:40.918320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:40.927012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:40.927715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:40.928005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:40.928290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:40.990549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:40.999332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.000805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.025616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.029503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a747e5-bb59-43e3-9b7f-8c570b0c6260 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.067718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.069783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.069925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.070203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.078667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.080121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:41.096922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:41.097845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:41.098739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 329/664] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:41.100286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.115856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 336/665] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:41.117069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.215064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.218979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400640b2-a5ca-4c3b-bffa-94a53b15a5e0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.223224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ab469-f8d6-4161-8b74-29fcf459f7ee None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.254678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.255286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.255576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.255967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.256953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:41.257273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.258839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.262241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.262918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.263204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.263483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.264225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:41.264533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.265900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.279188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 337/666] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:41.279188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.285712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 330/667] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:41.285712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.308735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.376276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7c-5560-49cb-8cb8-3795c8a8ca51 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.406685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3d199-21eb-49fb-82cf-737fbcd24ef5 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.410768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.411438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.411724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.412587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.413779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:41.414095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.415526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.428843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 338/668] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:41.429773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.450168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.450421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.451230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.451661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.452003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.463530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.465024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:41.486830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:41.487768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:41.488662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:41.488662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:41.488662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:41.488662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:41.488662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 331/669] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:41.489406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.541520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb287e1-9a25-404c-9c89-5db3f80af50d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.567038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.580063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.580866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.581178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.581454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.591584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.593079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.604120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14826e-5013-4cdb-b200-258945589eeb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:41.610724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:41.611582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:41.612455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:41.612455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:41.612455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:41.612455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:41.612455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 339/670] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:41.612721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.645133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.645784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.646069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.646464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.685952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.723441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.725156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.731533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e159-8694-4dff-a8ed-e60a2aa375a0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.767602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.768257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.768542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.768819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.769679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=8863fd88c1a2416c9fdb0148db03f6e5) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:41.769990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.771935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.788910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 340/671] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/role_assignments?user.id=8863fd88c1a2416c9fdb0148db03f6e5&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:41.788910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.814608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.860325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 332/672] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:41.861534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.915103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e66a-489c-45ab-bff8-616340d536d4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:41.948492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:41.955469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:41.956163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:41.956547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:41.956759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:41.957569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:41.957886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:41.959875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:41.974174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 341/673] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:41.975724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:41.995507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e8cc2-cd89-4408-b3a6-383009166528 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.038887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.039920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.040406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.040953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.041930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:42.042445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.044022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.057124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 333/674] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:42.057771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.091436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.102296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c455646-de64-42e4-ad41-bbf41b91e0a7 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.148048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.149120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.149410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.150407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.164196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.166211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.169607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae1d10-d3ca-43c0-9102-e2af6eee10c6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:42.189635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:42.190553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:42.191448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:42.191448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:42.191448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:42.191448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:42.192133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 342/675] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:42.192749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.212076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.213039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.213439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.213722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.221934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.223500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:42.240505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:42.242734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 334/676] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:42.245089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.248917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.308922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471844d-10f5-4851-816b-f05b125f9932 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.344659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.353137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.354238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.354666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.355093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] PATH_INFO: `/v3/users/swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.356055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec144241-ec1e-439a-bf96-bc6737f5dc5d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.363311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.365240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:42.381927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:42.383096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:42.384129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:42.390856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 343/677] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:42.392154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.393643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.394326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.394611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.394890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.396038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:42.396425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.398120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.411548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 335/678] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:42.412902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.507406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d259fb-f023-409c-99f2-d11e5eeb4acb None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.510723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.534121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b8b69-1713-449d-83b9-c7efad8fc659 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.545699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.546700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.547138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.547421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.548223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:42.548533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.550009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.566927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 344/679] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:42.566927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.576865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.577609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.577979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.578269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.579102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:42.579413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.580963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405900a7-6692-43cc-9491-54c9cc762d74 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 336/680] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a7ca-6a60-4ad6-85ef-1c89d6b6308a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a7ca-6a60-4ad6-85ef-1c89d6b6308a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a7ca-6a60-4ad6-85ef-1c89d6b6308a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 337/681] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:42.605030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758a7ca-6a60-4ad6-85ef-1c89d6b6308a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.616331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.682414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6207704-f0c7-4b92-ae10-8f0e551ad35f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.721841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758a7ca-6a60-4ad6-85ef-1c89d6b6308a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.722050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.722684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.722977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.723261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.734939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.736576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:42.753579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:42.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:42.755499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 345/682] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:42.756543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.766021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.766689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.766974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.767252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.768094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=ea2357b286df4e828b9f5024d71544ab) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:42.768408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:42.769880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1fee1-90e3-49b0-8a4f-91a332546039 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:42.782921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 338/683] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/role_assignments?user.id=ea2357b286df4e828b9f5024d71544ab&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:42.782921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.782921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.782921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:42.796211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:42.797485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:42.799179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.883535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 339/684] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:42.884641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:42.887255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:42.894712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301175b-7387-4ac9-946c-c7821e0888c2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:42.954400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:42.955266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:42.955651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:42.956029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.002500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfbe0d-c0ee-4a1a-a89d-46d86cb55f52 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.046144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.047863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.065181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.066120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.067476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.067476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.067965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:43.070158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.070498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.084633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.093128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 340/685] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:43.093736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.164586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.192584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 346/686] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:43.194304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.210561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415593-8fff-4010-8bc4-95647f1b2cab None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.247090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.247967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.248326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.248674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.261760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:43.270179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:43.271086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:43.272135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:43.277343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 341/687] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:43.277625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.320172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05dc42-0ea5-4e24-9b6d-51531fa55ce2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.344929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.357847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.358782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.359265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.359764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.360812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e1d08f8e3b4d4376aff855f4bce68c8c) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:43.361392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.363022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.369877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 347/688] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/role_assignments?user.id=e1d08f8e3b4d4376aff855f4bce68c8c&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:43.370925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.401637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6def1-8422-4430-a0d6-001f41e75c04 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.430483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.444676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.445485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.445775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.446102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.458200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.459976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:43.480266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:43.481266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:43.482126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:43.482126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:43.482126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:43.482126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:43.482126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 342/689] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:43.483778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56996a2d-91c5-435e-bef1-ecd8d4f30d42 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.487826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.521858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.523132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.523633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.524110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.525368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:43.525909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.527535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.537356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 348/690] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:43.544921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.641097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.657418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e94b16-dc0a-41d2-a8de-5a38eb34ed7c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.708072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b663a1-75ac-475e-8702-3ffeac34d558 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.717541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.718238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.718516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.718797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.719540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:43.719857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.725748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.740506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 343/691] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:43.747374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.764256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.765890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.766452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.766948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.781109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.783021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.793871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:43.806729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:43.807691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:43.808663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:43.808663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:43.808663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:43.808663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:43.811087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 349/692] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:43.816773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:43.880078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3a42e-693f-4988-8749-b9d22aa2376b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:43.932795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:43.934413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:43.934935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:43.935414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:43.943709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:43.945674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:43.961716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:43.962599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:43.963471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:43.966594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 344/693] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:43.972942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.014792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08876d92-c17a-4c42-b1d0-ee568b6fefe3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.016619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.083392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.093013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412bdf1-e2eb-4317-a615-32454dab114f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.101590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.102385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.102567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.102848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.103706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:44.104019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.105565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.128375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 350/694] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:44.138271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.139209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.139696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.140191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.141238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:44.141752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.143377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.147833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.159444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 345/695] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:44.166482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.274934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0be67e-077c-4f92-a916-559819baf512 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.281207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd940db5-6382-423d-9020-4406abacf6f1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.312823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.313706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.313995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.314271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.320842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.321843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.322334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.322813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.323400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.324088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e57e080ca56d4f28bb1692d147cf4caa) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:44.324618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.324869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.326704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.333849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 346/696] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/role_assignments?user.id=e57e080ca56d4f28bb1692d147cf4caa&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-212fa0ea-fe56-47d9-a0a1-3ef6ca585216 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:44.341095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:44.342113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:44.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 351/697] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:44.348159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.354288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ebd99-1beb-4f0f-b0eb-67d9dba30021 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.354586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ebd99-1beb-4f0f-b0eb-67d9dba30021 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.354868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ebd99-1beb-4f0f-b0eb-67d9dba30021 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.356480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 352/698] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:44.365940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.365940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.365940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.390289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.400411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:44.401660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:44.447059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.518518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9e304-2cf0-47ab-bf79-6a0416408f7a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.565456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.566503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.567000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.567389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.569532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:44.570065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.571701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.572353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 353/699] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:44.590550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 347/700] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:44.595455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.599239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.712195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.712940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8594d1-a20b-469c-9621-f9e8cce33193 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.746514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2c1f9-10c4-4a0b-b2bf-0a96a443a1f0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.753598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.754646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.755097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.755527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.756821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.758328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:44.773594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:44.775010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:44.776669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 354/701] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:44.780956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:44.791992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.793544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.794183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.794669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.806224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:44.896036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.898379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.910052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab39d6-ffc1-43a0-b612-4efc99595ae0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:44.969737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:44.970495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:44.970790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:44.971076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:44.971907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:44.972226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:44.973699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:44.988094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 355/702] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:44.994713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.007616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:45.041296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 348/703] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 00:14:45.045300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.187030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:45.215620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079ef6d-c513-4d21-a556-ebea06cf00be None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.258516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ca7c5-6bae-41d1-af8b-e65b4ce0d350 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.274895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:45.276361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.276977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.277467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.278434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:45.278964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:45.280837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce026dae-2dae-4e4e-b387-a71cf751a7c8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:45.294946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 356/704] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:45.307387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:45.308620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.309142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.309656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.324448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3457-b5df-4e5d-bfee-e925f1e63657 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.324783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3457-b5df-4e5d-bfee-e925f1e63657 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.325058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af3457-b5df-4e5d-bfee-e925f1e63657 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.326778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 357/705] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:45.328850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:45.330725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:45.336911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.336911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.336911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:45.367592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:45.368553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:45.369465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:45.369465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:45.369465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:45.369465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:45.370972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:45.372367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:45.381157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 349/706] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:45.391678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.449994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:45.508492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 358/707] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:45.541803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.553949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c8a2-4b3b-4cef-a1da-219c6fc68598 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.592763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:45.593475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.593767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.594073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.594909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:45.595230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:45.596616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:45.613721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 350/708] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:45.618634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.638829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:45.666564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf79237-2c5d-4a80-8f80-10314735077e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.703706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:45.706836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.706836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.706836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.717518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:45.720058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:45.745102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:45.746240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:45.747369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:45.747369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:45.747369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:45.747369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:45.748859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 359/709] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:45.752054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:45.754716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.769540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b56ec-9edf-436a-825a-32d24ba0362c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.832615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:45.833707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:45.833968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:45.834249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:45.847292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:45.848742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:45.872129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:45.873139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:45.874291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 351/710] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:45.888133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:45.939932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906ca45-5e79-40c8-8592-14b919881f9d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:45.989339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:46.007084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.008500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.008801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.009123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.009876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:46.010196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.011597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.039627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 360/711] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:46.042924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:46.083267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b27f-1b86-469e-bce7-f68078f9a741 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:46.157789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.158482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.158767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.159046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.159860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:46.160172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.161689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.179543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 352/712] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:46.185827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:46.207080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:46.237666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685d12-e3d5-496d-bb03-019744129d4f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:46.292531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.293413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.294100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.294367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.295085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.297078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ec7b1-acc1-4a4a-b246-3efb1698f10e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.309250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99706d-b955-42c3-8568-4cfef7ae6ce8 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:46.325139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 361/713] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:14:46 2022] POST /identity/v3/users => generated 254 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:46.327289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2450-994e-4c23-9e0c-c81c1ff6a9cf None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.327587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2450-994e-4c23-9e0c-c81c1ff6a9cf None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.327895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b2450-994e-4c23-9e0c-c81c1ff6a9cf None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.330105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 362/714] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:46.339961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.340463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.349092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.363516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.364202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.364494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.364778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.367606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:46.367933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.372573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.388937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 353/715] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:46 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:46.398571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:46.399930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:46.456221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:46.510541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 363/716] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:46.550730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:46.627164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:46.669298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f0658-ffce-408f-a4b6-630807e02bc0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:46.718214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.719040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.719320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.719599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] PATH_INFO: `/v3/services/network` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.720264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.721735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:46.736253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:46.738529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:46.740440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 354/717] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:46 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:46.747395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:46.903663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f004ac-8578-4cdf-840b-0c38c16a3231 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:46.947290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:46.947940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:46.948231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:46.948511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:46.949253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] RBAC: Adding query filter params (name=network) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:46.949560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:46.950936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f433b-076c-422d-a65e-41ebfba78c8d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:46.955753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 364/718] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:46.961206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:47.105692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20936-d2c0-404c-a062-4ad4373fd254 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:47.157658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:47.158257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:47.158544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:47.158825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:47.159539 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] RBAC: Adding query filter params (type=network) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:47.159852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:47.161358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3f00-7975-4950-b98f-86b18f1aefb0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:47.165045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:47.168903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 355/719] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:47.385856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9900-4927-40a2-ba6a-c67723c39244 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:47.386238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9900-4927-40a2-ba6a-c67723c39244 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:47.386422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e9900-4927-40a2-ba6a-c67723c39244 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:47.388114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 365/720] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:47.394313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:47.395243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:47.395243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:47.419929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:47.419929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:47.500682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 356/721] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:47.525933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e9900-4927-40a2-ba6a-c67723c39244 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:47.566007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:47.649614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e9900-4927-40a2-ba6a-c67723c39244 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:47.692001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:47.692730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:47.693672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:47.693992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:47.715943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:47.729728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:47.730625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:47.732125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 366/722] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:47.737986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:47.925604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a7aa7-3d5c-4ac7-9988-7b9b1a89bb85 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:47.969510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:47.982912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:47.983607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:47.983898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:47.984182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:47.984973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:47.985288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:47.986708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fc284-ed4b-40e9-b08c-bc795bd5357a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.004574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 357/723] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:48.010489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.086123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec46-d554-4d1a-832c-96b971817b13 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.086442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec46-d554-4d1a-832c-96b971817b13 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.086691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec46-d554-4d1a-832c-96b971817b13 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.088059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 358/724] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:48.093111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.093406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.093683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.126642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:48.127836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:48.141380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:48.149306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c68475-6a3e-417a-8c30-fd025d5249e4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:48.209694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:48.210515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.210804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.211083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.233079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:48.234478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.256716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 359/725] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:48.264918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:48.268309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:48.270375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:48.270375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:48.270375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:48.270375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:48.270375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 367/726] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:48.276925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.299974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.438551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93516c-fae8-4af4-922e-80d8d9debd25 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:48.462498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:48.477677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:48.478581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.479170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.479596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.480611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:48.481141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:48.482856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.496690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 360/727] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:48.501734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.509063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a3fbd-2273-4f1b-b0af-4dfec9931291 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:48.597609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:48.598294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.598582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.598863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.599647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:48.601211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.616619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae626d60-8013-467f-b00c-f32a016db7b3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:48.627803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 368/728] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:14:48 2022] POST /identity/v3/services => generated 251 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:48.668749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:48.669608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.669898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.670179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.679491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:48.681046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:48.697456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:48.698495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:48.699473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 361/729] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:48.704593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.729720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:48.872449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0d1e1-0db0-4c49-b01d-c194840c76da None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:48.923121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:48.923806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.924091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.924369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.925239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:48.925551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:48.927063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7983-ecdb-44c3-a41d-ad9dfb0c524f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:48.948760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 369/730] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:14:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:48.954391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:48.958679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66218-7760-480a-a35b-8fae4a97d659 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.959391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66218-7760-480a-a35b-8fae4a97d659 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.959391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb66218-7760-480a-a35b-8fae4a97d659 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.960708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 370/731] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:14:48.967123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:48.967413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:48.967690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:48.991839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:48.993687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:49.019382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.082949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.094168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 371/732] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 00:14:49.101466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b882b-142a-49c6-a401-b9680fae47d8 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.119183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.151151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.151742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.152030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.152313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.239022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:49.241080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:49.282080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.353374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 362/733] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 00:14:49.359190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.377065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560926f5-25cd-4e94-b554-c09587f76663 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.427899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.428994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.429293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.429578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.443348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d85bceaf-1e09-4b87-b7c2-9537c4c24de7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:49.453634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:49.454587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:49.455392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:49.457943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 372/734] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:49.460159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa914-7c3c-4162-9e6e-a47372813d63 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.460656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa914-7c3c-4162-9e6e-a47372813d63 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.461175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fa914-7c3c-4162-9e6e-a47372813d63 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.462929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 373/735] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:49.470383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fa914-7c3c-4162-9e6e-a47372813d63 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.492794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa290c-4a57-4f8a-b1b6-2e0bbc41e386 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.529351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.530432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.530897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.531340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.539272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.544142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:49.546020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:49.557056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-e5273e51-aa36-4a74-bfed-3feda926c740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:49.567217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:49.568660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:49.569583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:49.569583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:49.569583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:49.569583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:49.571778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 363/736] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:49.573237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.573725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.574215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.591602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:49.592910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:49.606219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fa914-7c3c-4162-9e6e-a47372813d63 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.645841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.646715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.647098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.647389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.648158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:49.648472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:49.649888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:49.664227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 374/737] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:49.665266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.666701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 364/738] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:49.667600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.776695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c53bd4-65c8-4411-b2e0-b0361fef0afc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.797198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a315-3d78-45cc-af11-e12b3e5e0eb9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.799706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.817470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.818815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:49.818970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.819174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.819744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.831112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:49.832584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:49.840583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.841573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.842071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.842540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:49.843474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:49.843978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:49.845584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:49.851541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:49.852408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:49.853321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:49.853321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:49.853321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:49.853321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:49.853321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 365/739] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:49.853989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.860863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 375/740] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:49.861967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:49.962561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad22c3-9b65-4966-bbe0-52ad09caedb4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.965770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4278-afe9-4ffb-abae-2f3ae48470b3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:49.998203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:49.998923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:49.999206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:49.999481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.000838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.001577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.001896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.002183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.002920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:50.003235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.004606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346a454-b5c0-4f17-a5c0-90aeffcf8e2e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.010485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:50.024135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 376/741] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:50.024319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:50.025901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:50.026760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:50.027386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486534-24e1-4cdc-8c2e-cb4604b0fd50 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.027386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486534-24e1-4cdc-8c2e-cb4604b0fd50 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.027386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486534-24e1-4cdc-8c2e-cb4604b0fd50 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.027560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 366/742] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:50.027560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.027678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 377/743] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:50.027678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d486534-24e1-4cdc-8c2e-cb4604b0fd50 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.058604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.071187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.142269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545f353-4979-4602-8fb8-99fdd40985e2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.151212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d486534-24e1-4cdc-8c2e-cb4604b0fd50 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.178570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.179408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.179703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.179986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.187239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.188697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.190426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.191180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.191471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.191745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.199164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.200590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:50.204311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:50.205213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:50.206319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:50.208415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 367/744] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:50.210051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-dd565f6d-8617-4cbb-ad03-4baab9126715 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:50.215508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:50.216629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:50.217810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:50.221050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 378/745] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:50.222333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.222622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.222902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.239767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:50.241147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:50.311575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 379/746] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 00:14:50.312420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.324792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.336040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0500c-8985-461f-871d-304c59aa4355 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.350081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.372543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.373681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.373976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.374258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.375027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:50.375331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.376927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.390418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 368/747] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:50.390418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.436034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919f07d-4428-4c43-b605-ef439c9b5e69 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.473233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.473912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.474204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.474481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.475398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:50.475714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.477210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.488934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 380/748] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:50.490226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.518411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2a46a-1860-4279-ab9b-1f779a76a592 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.573473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.574615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.575135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.575589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.576503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:50.576945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.578761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.594170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.601289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 369/749] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:50.601990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.617852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d591532-9915-4772-925b-cf96a0d88418 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.622359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.667330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.668017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.668308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.668705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.669490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.670876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.696916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 381/750] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:14:50 2022] POST /identity/v3/services => generated 240 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:50.696916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.754780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dce1b-9246-4fd9-9ea3-62686fc7df90 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.820260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.821666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.822174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.822998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.838142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.839940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.840543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde19a3-2591-4544-b63a-16e170c1fa01 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:50.861733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:50.862683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:50.863561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:50.863561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:50.863561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:50.863561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:50.865992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 370/751] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:50.867179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.879701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:50.880978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:50.881314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:50.881636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:50.882501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e57e080ca56d4f28bb1692d147cf4caa) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:50.882837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:50.884319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:50.890941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 382/752] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/role_assignments?user.id=e57e080ca56d4f28bb1692d147cf4caa&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:50.891724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:50.912589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.936026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:50.985417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852fb8e-000b-4576-bfcf-72182763661d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.045850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa1658-8d28-4b8c-a940-0c17837ffe9a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.048098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.048954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.049242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.049517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.070189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.071662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:51.101990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:51.102914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:51.103806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:51.103806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:51.103806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:51.103806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:51.103806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 371/753] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:51.109703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.120240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.120938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.121226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.121507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.122239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:51.122552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.123925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.139396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 383/754] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:51.144670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.243599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe2f93-1b96-424f-afc0-b8a8b17ade7f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.259527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.263213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.298457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.299140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.299433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.299718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.300456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:51.300771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.302741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3429d-52c6-4098-958f-76bed4c036a9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.306254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a4c7e-f285-47d0-a043-390c9ca3537a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.318843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 372/755] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:51.320815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3d14-e0fb-4fc8-ad6c-215e75ce6404 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.321306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3d14-e0fb-4fc8-ad6c-215e75ce6404 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.321757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3d14-e0fb-4fc8-ad6c-215e75ce6404 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.324366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 373/756] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:51.325612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c3d14-e0fb-4fc8-ad6c-215e75ce6404 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.348581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.350322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.350685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.351040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.359460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.361005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-97eea975-45fa-4dd1-9a61-7309c25668ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:51.377418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:51.378426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:51.379444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 384/757] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:51.381320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.381729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.382141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.399920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:51.401746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:51.465377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c3d14-e0fb-4fc8-ad6c-215e75ce6404 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.478107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 385/758] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:51.479278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.525457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.526380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.526673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.526956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.534483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.536070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:51.551653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:51.553757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:51.554646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 374/759] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:51.555122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.559910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.589753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.614643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40cafd-e95f-46cd-9de5-ace1fcef64bb None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.661375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.663421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.664134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.664588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.665891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:51.666782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.668652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.669178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ffa3-ce37-47c9-8781-64af5c198a98 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.692718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 386/760] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:51.695092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.714317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.715232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.715765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.715873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.716509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.717929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:51.738078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:51.739022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:51.740417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:51.744825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 375/761] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:51.745761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.835741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd6804-8a7d-4620-84f8-5ace3a597ae4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.861521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.864482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:51.869160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb3953-7db9-46d7-981e-5f588387c7ba None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:51.882992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.883945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.884387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.884809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.885770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:51.886247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.887834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:51.901979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 387/762] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:51.903239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:51.905417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:51.906165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:51.906521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:51.906943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:51.981126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:51.983367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.016480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd87a5-3c4f-4896-a6be-9d4bf330b45d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.053864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.054813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.055248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.055672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.056567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:52.057055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.058580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.064120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 388/763] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:52.065089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.087558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:52.094762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 376/764] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:14:52.095592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.119320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:52.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00c061-44dd-4030-86fa-bba295b42a23 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.209129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f59f-d321-45e1-b401-ecdd3ed48b13 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.220651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.221828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.222308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.222729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.226086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=d50db388db1446eeb60ccaa17ff55389, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.227696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.246087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 389/765] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:52 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:52.246481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.247159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.247450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.247673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.247783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.248491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:52.248841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.250254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.261773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 377/766] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:52.262722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.341332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:52.400900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c8ea0-886a-4438-a7a9-ab724741ba33 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.419510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc28912-b7fe-4217-994c-3a8ff93ce878 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.426999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:52.452364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.453504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.453940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.454364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.455581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] RBAC: Adding query filter params (interface=public, service_id=c49a00f5c3f24e3b93babdfd7f6f9b22, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:52.456095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.457646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.468233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 378/767] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/endpoints?service_id=c49a00f5c3f24e3b93babdfd7f6f9b22&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:52.477915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.508073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.508929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.509228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.509514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.533305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.535030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:52.555445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:52.556633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:52.557671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:52.557671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:52.557671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:52.557671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:52.559495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 390/768] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:52.564767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.616014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3aacc-b70b-4bff-aa37-efbb1096c67e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.643253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:52.656476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12048493-9012-4220-830d-269392eba8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.657112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048493-9012-4220-830d-269392eba8f3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.657405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048493-9012-4220-830d-269392eba8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.657691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048493-9012-4220-830d-269392eba8f3 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.658393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12048493-9012-4220-830d-269392eba8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.659772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12048493-9012-4220-830d-269392eba8f3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.666102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 379/769] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:52.694280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eb972-b694-4da3-b1e1-aa893cd72d1c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.731741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.732423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.732711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.733025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.733763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:52.734079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.735459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.749067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 391/770] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:52.759106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12048493-9012-4220-830d-269392eba8f3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.883350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12048493-9012-4220-830d-269392eba8f3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:52.923340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:52.924080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:52.924371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:52.924655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:52.932562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:52.934050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c538e836-48f2-4c88-9c12-ca0062be58a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:52.949673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:52.950587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:52.951460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 380/771] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:52.957028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:52.981261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.069313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4cf02-22f9-4ab2-95ac-57cf9efebe24 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.110596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.111332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.111627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.111910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.112722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:53.113080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:53.114535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:53.119836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6aee-200b-4f7f-b73b-f3122acddf1a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.120132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6aee-200b-4f7f-b73b-f3122acddf1a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.120418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b6aee-200b-4f7f-b73b-f3122acddf1a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.121826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 381/772] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:53.126921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.127224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.127504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.132820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 392/773] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:53.143576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.162421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:53.163785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:53.247818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.293314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.297000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 382/774] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:53.322191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382a51-defd-4cf3-8b60-f61f96f048a4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.328608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.385669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.386899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.387259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.387610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.389208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d58b01886c5d441da40130bffc817da7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:53.389599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:53.391816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c39b5a-abe4-42c8-9f19-1c1eb0b651be None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:53.400522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 393/775] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/role_assignments?user.id=d58b01886c5d441da40130bffc817da7&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:53.409872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ec0cd-b51d-4025-aedb-28f9a6b1ded7 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.410221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ec0cd-b51d-4025-aedb-28f9a6b1ded7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.410579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ec0cd-b51d-4025-aedb-28f9a6b1ded7 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.412823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 394/776] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:53.420380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.421060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.421060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.448288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:53.449524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:53.518150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf250e8-b96e-47f8-9505-bc303defc58c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.559741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.560547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.560829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.561130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] PATH_INFO: `/v3/services/network` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.561791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:53.563166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:53.575716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:53.577986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:53.579932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 383/777] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:53.590286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 395/778] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:14:53.590434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.632474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.655000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.717528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b64a1e-15ba-472a-9937-93ceab7c7dee None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.753686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.754406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.754696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.754974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.756226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] RBAC: Adding query filter params (name=network) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:53.756542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:53.757921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:53.763040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 384/779] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:53.768768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.794989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.862201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01833fb-8e83-494e-a08b-6abd6831f7c4 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.895269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bba8b-e89c-4938-96e3-0e31e555bd47 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:53.903350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:53.930322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.931148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.931437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.931715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.933657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:53.934352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:53.934627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:53.934905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:53.935641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] RBAC: Adding query filter params (type=network) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:53.935953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:53.937389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:53.943716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 385/780] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:53.949893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:53.953469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:53.969603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:53.970585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:53.971393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 396/781] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:53.980422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:54.132972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6bb4c-2c75-423e-ab29-4c0912877cdc None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.155802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225c61-1b49-44bf-9e5a-32aebd843ad9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.168038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.168713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.169041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.169457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.170327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] RBAC: Adding query filter params (interface=public, service_id=a7fc543ea7c74f3d94410f1ef61f09dd, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:54.170721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.172167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.178504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 386/782] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/endpoints?service_id=a7fc543ea7c74f3d94410f1ef61f09dd&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:54.183487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:54.209242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:54.218292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.218988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.219282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.219565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.220582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:54.221368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:54.221701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.223191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.233469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 397/783] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:54.239329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:54.339344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f51e8-3fbe-42e2-aecb-f031b1efb5d2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.374130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.375166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.375649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.376122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.377024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.378578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.384507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 387/784] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:14:54 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:54.440525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc011-1d09-40ec-a20f-c5115898d41c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.500991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.501842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.502134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.502418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.516844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.518507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:54.545174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:54.546129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:54.546995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:54.546995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:54.546995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:54.546995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:54.546995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 398/785] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:54.552434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:54.697347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:54.701211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d6639-d8a0-4a1a-a680-94b3084ef309 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.749321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.749998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.750292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.750570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.751304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:54.751619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.753011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.766842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 388/786] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:54.772575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:54.820520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:54.902207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f48e36-4d91-4e19-8943-960dff063b91 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:54.947595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:54.948326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:54.948617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:54.948920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:54.962006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:54.963462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-071e0962-3906-49b0-afac-9cbea9130487 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:54.980927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:54.981908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:54.982840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 399/787] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:54.988023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.109125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0564ae-3e0e-4f5f-9670-7c4311ac93c7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.158621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.159306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.159593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.159875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.160728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:55.161073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:55.162580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:55.171481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb3d8-00da-4038-a129-b4d88206a532 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.171767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb3d8-00da-4038-a129-b4d88206a532 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.172041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdb3d8-00da-4038-a129-b4d88206a532 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.173744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 400/788] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:55.179837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.180377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.180893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.185622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 389/789] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:14:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:55.191000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.217960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:55.219370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:55.237251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:55.316134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 401/790] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:55.344955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.348606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71524e91-8b4e-4288-aed0-75bc4d3859cb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.402663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:55.413641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.414913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.415393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.415871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.427456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:55.429319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f0aa3458-e8fe-4ec8-bf44-0cf10b3c8dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:55.450315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:55.451218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:55.452088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:55.452088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:55.452088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:55.452088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:55.454546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 390/791] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:55.456088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e0d6-3fe1-4119-987f-15e9c4348d0a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.456600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e0d6-3fe1-4119-987f-15e9c4348d0a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.457113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e0d6-3fe1-4119-987f-15e9c4348d0a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.463312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 391/792] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:55.464451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e0d6-3fe1-4119-987f-15e9c4348d0a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.486740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220e0ef-7100-432b-976c-2db67184a003 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.532169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.534008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.534522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.535253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.541547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:55.551997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-76fcf19e-b80a-4c95-8cc0-5b8b3280ef2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:55.563263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:55.564188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:55.565052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:55.568617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 402/793] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:55.570865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.571606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.572104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.581311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6e0d6-3fe1-4119-987f-15e9c4348d0a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.594584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:55.596975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:55.630166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.632031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.632539 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.633272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.635061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:55.635594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:55.638156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:55.654482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 392/794] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:55.655898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.682274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 403/795] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:55.683462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.730808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:55.794745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd171b11-8b14-435c-b037-7becea12161c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.820357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00c92f-e50a-4bd6-a24a-0f48e493bf3c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.825562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:55.834964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.835997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.836463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.836980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.837942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:55.838467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:55.840081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:55.849684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 393/796] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:55.850777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.862212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:55.863018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:55.863304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:55.863579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:55.872083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:55.873563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:55.887221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:55.888138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:55.889476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 404/797] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:55.890186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:55.963757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875aa71c-8456-4ff0-8766-936121e289fb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:55.999591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.000434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.000719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.001032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.001709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.003119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8b585dd2-9c37-4ac0-9b76-243542eb8e39 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:56.016576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:56.017688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 00:14:56.018529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:56.025313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 394/798] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:56.025313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b69-b351-450b-8d56-206c2ea7bbbc None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.025313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b69-b351-450b-8d56-206c2ea7bbbc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.025313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55506b69-b351-450b-8d56-206c2ea7bbbc None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.025313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 395/799] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:56.026378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506b69-b351-450b-8d56-206c2ea7bbbc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.090939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.113344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.137773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31a985-afce-4207-9e48-18d520105027 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.141273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55506b69-b351-450b-8d56-206c2ea7bbbc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.172859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.173683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.173967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.174245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.178603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.179268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.179552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.179830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.180743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:56.181090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.182643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.184278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.185720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.200661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 396/800] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:56.200661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b00aa8a1-7e21-4108-ae34-7710d4c74ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:56.203256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:56.204116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:56.205061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:56.205061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:56.205061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:56.205061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:56.205061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 405/801] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:56.205727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.205951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.206228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.222906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:56.224117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:56.290657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 406/802] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:56.291466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.316189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a00223-e1f8-4ef1-9b6f-99d2d36c751a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.334263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.357198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.357800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.358493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.358778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.359498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:56.359811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.361228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.366304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 397/803] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:56.367126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.375946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.414448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35365bf9-5b84-4f27-af82-ea1ece182a37 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.451336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.452802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.453122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.453401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e57e080ca56d4f28bb1692d147cf4caa/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.455852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=e57e080ca56d4f28bb1692d147cf4caa, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.457317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.476938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 407/804] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:56 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/e57e080ca56d4f28bb1692d147cf4caa/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:56.476938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.487595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cbc8f-feb4-48d9-a7e6-fef31be6b82a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.535990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.537277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.538073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.538433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.539275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:56.539672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.541884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.560901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 398/805] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:56.561879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.611519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.613854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d20c30-54d8-4d0b-8d28-c52b09d9fb00 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.653551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.665960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.667115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.667473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.667826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.680553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:56.688407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:56.689398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:14:56.690245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:56.690873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2ea4d-5edf-4947-b23e-6f0f3ee47d08 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.697036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 408/806] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:56.697036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.748123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.749593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.750349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.750865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.752098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:56.753035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.755676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.762700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 399/807] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:56.763519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.813911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026094c-3243-4f60-ae19-bb416b0e5535 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.860134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.861010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.861305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.861646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.871312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.873535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:56.894080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:56.895055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:14:56.896625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:56.903430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 409/808] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:56.903430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.906912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412498f6-77f4-49d9-ad30-2eefc5094909 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:56.908740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:56.948702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:56.949489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:56.949783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:56.950068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:56.951059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:56.951168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:56.952523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:56.962611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 400/809] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:56.963413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:56.971464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.036985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3d203-0797-4277-930b-6144b6202985 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.071776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.072852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.073402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.073880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.074878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.075330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b808b3-6d7b-4ecb-80fd-ad10b873d987 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.076775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.105654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 410/810] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:14:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:57.107049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.154399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.155214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.155503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.155780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.156648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:57.158396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.158537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fbed4-1b20-42b1-b155-08dadd18c084 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.179815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 401/811] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:14:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:57.181207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d526-1214-4ada-b1a1-eaa4220ec3d0 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.181501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d526-1214-4ada-b1a1-eaa4220ec3d0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.181784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d526-1214-4ada-b1a1-eaa4220ec3d0 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.183315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 402/812] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:57.185428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1d526-1214-4ada-b1a1-eaa4220ec3d0 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.206236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.235420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819ff9f-cc66-4192-b4d1-c2ff0618172b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.237599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.279892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.281019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.281543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.281995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.295398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.297237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.299459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1d526-1214-4ada-b1a1-eaa4220ec3d0 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-38731a06-ebae-4c03-962b-767a0250e81b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:57.318061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:57.319027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:57.320621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:57.320621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:57.320621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:57.320621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:57.320621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 411/813] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:57.321109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.321335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.321615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.341205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:57.343293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:57.349083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.349757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.350045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.350324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.425714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.427889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.433829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 412/814] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:57.433829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.469289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.536008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.556111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8207eb-6f42-475a-b3b5-f475b223757b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.575984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 403/815] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:14:57.576854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.594594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.595761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.596623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.597680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.600577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:57.601255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.602852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.621770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 413/816] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:57.623372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.691333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c893c-6438-47e2-853e-4246595f7c6f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.710928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.734228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.735331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.735822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.736302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] PATH_INFO: `/v3/services/network` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.737240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.738832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.743855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1ecb6-9841-4e77-adab-53fa6495f068 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:57.752195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:57.753245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 00:14:57.754075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:57.762199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 404/817] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:57.763343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.782612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.783515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.783809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.784099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.795920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.797522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.800218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:57.818558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:57.819522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 414/818] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:57.821129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.876390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254f354-1370-418b-a33b-2328d7ce25e4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.912551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.914285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.914800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.915281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.925207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.927500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.930933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da3ada-7349-4a47-af4c-f1f953f7d23e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:14:57.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:57.945619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:14:57.946545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:57.949160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 405/819] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:57.950231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:57.971919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:57.972597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:57.972907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:57.973197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:57.973971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] RBAC: Adding query filter params (name=network) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:57.974286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:57.975752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:57.982202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 415/820] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:57.982202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.005229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.048482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.069746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfc279-ee51-425f-bab9-1a2c2025975c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.093560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c3cf-5845-48f1-b9d8-a3a24d941885 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.106990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.108101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.108668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.109227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.110185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.110702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.112561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.126262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 406/821] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.127467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.134372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.135061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.135348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.135627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.136474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.136789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.138252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.162829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 416/822] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:14:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.163741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.244634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084f889-56bc-4a13-a0f0-32cf3a126c7d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.246586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.278290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b365e-1fda-4b7a-9491-46813c2f34b0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.286576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.287635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.288183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.288686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.289767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] RBAC: Adding query filter params (type=network) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.290285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.292025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.296432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.304326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 407/823] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.305900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.314390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.315194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.315479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.315758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.323949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.325550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:58.341159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:58.342146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:58.343548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 417/824] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:58.348254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.439388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5d8a3-a5c4-42a9-824a-ea7beaf33796 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.465172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517262c-34d0-4762-9b38-49dc908bb778 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.488107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.490587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.491068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.491782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.508563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.509813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.509813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.509813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.510490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.511501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.511963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.513468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.534260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 418/825] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:14:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:14:58.535487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:58.542143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:58.543098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:58.544040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:58.544040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:58.544040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:58.544040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:58.544040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 408/826] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:58.549452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.582036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.594739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.675320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16833-b97a-4535-9b94-bfc79a90a8d6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.722440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.723510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.724076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.724599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.725731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.726275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.728717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.744965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 419/827] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.753332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.787334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb692f4-c10b-4f54-8868-44ad8ecd7fcc None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.833591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.834393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.834795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.835087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.835826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.836145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.837555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b47cb-d259-440e-b3ed-f7280c38c7b7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.852159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 409/828] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.853573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee39131-ae52-4fab-9423-02e84e1493e7 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.853989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee39131-ae52-4fab-9423-02e84e1493e7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.854281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee39131-ae52-4fab-9423-02e84e1493e7 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.855659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 410/829] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:58.860790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee39131-ae52-4fab-9423-02e84e1493e7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:58.868856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:58.881432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b5e79-4c8f-403f-b4fe-a24f2c4d5df8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:58.925427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:58.926756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.927652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.928030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.929209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:58.929534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:58.930982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0a48d-9481-4440-a10a-b8c9be7fd38c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:58.937506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 420/830] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:58.938795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:58.939083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:58.939366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:58.957796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:58.959129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:58.977552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.002499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee39131-ae52-4fab-9423-02e84e1493e7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.046291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 421/831] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:59.055039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.056240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.056746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.057259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.065045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.066901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.073316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:14:59.093586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:59.094630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:59.095553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 411/832] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:59.102503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.183253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.203388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3706d-ed7a-4a70-999d-8906d03e78bc None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.241732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d624f6-fc0f-4bc7-abee-d05d85160d3f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.252453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.254280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.254785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.255555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.270411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:14:59.278570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.279404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.279796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.280175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.281105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:59.281532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:14:59.282513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:59.283527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:14:59.284562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:59.285751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.287336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 422/833] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:59.293126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.296192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.304008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 412/834] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:59.313838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.417520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2cc1f-2761-4c73-a904-b1080b6d47d4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.444094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4595a6-d5b7-41de-a753-e9329f628c87 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.466651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.468060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.468797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.469245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.470406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:59.470889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.473493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502caed-b35c-4e4b-be88-e8c81a63d3f7 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.484144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 423/835] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:14:59.484704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.485796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.486041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d967e5-460e-4dd4-a93c-f685dd007d12 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.486273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.486628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d967e5-460e-4dd4-a93c-f685dd007d12 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.486739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d58b01886c5d441da40130bffc817da7/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.487135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d967e5-460e-4dd4-a93c-f685dd007d12 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.488756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 424/836] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:14:59.490388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d967e5-460e-4dd4-a93c-f685dd007d12 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.499957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.506104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=d58b01886c5d441da40130bffc817da7, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.507882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215fbf7-4821-4222-ba30-511100ff790b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.524313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 413/837] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:14:59 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d58b01886c5d441da40130bffc817da7/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:14:59.526189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.526481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.526767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.544655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:14:59.546057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:14:59.607300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.623245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d967e5-460e-4dd4-a93c-f685dd007d12 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.646921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 414/838] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:14:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:14:59.670641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.676564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.677701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.677988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.678268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.690507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.691997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:14:59.711668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:59.712600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:59.718654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:14:59.718654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:14:59.718654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:14:59.718654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:14:59.718654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 425/839] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:14:59.720692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.809490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dfbca-644b-4fc0-8aec-fcabf7b71f27 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.811521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:14:59.855887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.856858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.857181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.857523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.858302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.860276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.861992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4cc41-3a5f-4c66-978c-37ed984cd307 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:14:59.876247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:14:59.877497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:14:59.878412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 415/840] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:14:59.885314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.907056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:14:59.908066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:14:59.908501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:14:59.909040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:14:59.909950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:14:59.910408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:14:59.912796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:14:59.931058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 426/841] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:14:59.932538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:14:59.940820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.014125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d984bdc-d070-41bf-90f3-8cfb7e67c42b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.044485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0617eb6-0fdc-4867-9475-b069703ec2d8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.057672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.058638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.058934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.059219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.059945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:00.060261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.062228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.066150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.073166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 416/842] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:00.076410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:00.093870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.095069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.095368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.095653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.105643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.108087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:00.127787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:00.129007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:00.134624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 427/843] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:14:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:00.135202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:00.204372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d0443-26cc-4208-821c-cbeaf568b70d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.212546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.251090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c710db1-e35b-4a6a-8030-c4b00d6c8601 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.253767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.256244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.256642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.257055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.257890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:00.258308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.259789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.272911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 417/844] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:00.296843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.297769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.298058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.298384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.299249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:00.299561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.301107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.323105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 428/845] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Jun 16 00:15:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:00.328746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:00.443631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab21b-22db-4215-bc70-b4301ae3c321 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.481561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.482224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.482566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.482850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.524029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.559220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.560911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.666368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 418/846] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:15:00.671945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:00.711029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.784135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb167b-cd3d-4aa4-93fe-847beee42d32 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:00.824354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:00.825207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:00.825496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:00.825772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:00.840023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:00.841602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:00.859737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:00.860641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:00.861558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:00.861558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:00.861558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:00.861558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:00.861558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 429/847] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:00.867189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:00.943587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:00.977304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb810d94-a7f7-4d5b-bba2-76c574f42717 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.013233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.013916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.014206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.014750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.015209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:01.015526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.016921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.030109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 419/848] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:01.035589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.126676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:01.140332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79b7e-2170-4f29-a5a1-c56409c6d26c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.177176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.178271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.178623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.178968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.186229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.187746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-55b002a8-7ce7-45f0-80a4-766d19919429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:01.202723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:01.203871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:01.204960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:01.209030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 430/849] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:01.212284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.275998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b96a-22bd-43dd-99a1-0453e489e843 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.276178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b96a-22bd-43dd-99a1-0453e489e843 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.276463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122b96a-22bd-43dd-99a1-0453e489e843 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.278114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 431/850] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:01.282993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.283291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.283574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.300980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:01.302283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:01.325456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c0db4-b99e-4de8-834f-f93dfd7c8a4b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.356288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:01.363205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.364809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.365517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.365978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.367407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:01.368137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.369808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.381134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 432/851] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:01.384370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 420/852] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:01.389967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.397920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.493444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:01.514221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df30c1a-112f-4108-affb-82f4141dc90a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.527573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c90c8-d332-4e37-baba-7128a034a3ae None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.555014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.556159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.556592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.557061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.557891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.559405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.565649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.566388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.566756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.567099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.568338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=e57e080ca56d4f28bb1692d147cf4caa) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:01.568338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.569911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:01.574412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:01.575509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:01.576535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:01.584913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 433/853] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/role_assignments?user.id=e57e080ca56d4f28bb1692d147cf4caa&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:01.585096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 421/854] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:01.588253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.703300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d0a1b-1467-43ff-a7b5-5eb279681c19 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.755282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.756630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.756630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.756768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.757499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:01.757815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.759260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1a659-f255-4766-86b5-f6801be5b7c1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.765299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 434/855] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:01.771843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:01.824412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192de73-b70e-4a47-88e5-dd322197e889 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.824728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192de73-b70e-4a47-88e5-dd322197e889 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.825009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192de73-b70e-4a47-88e5-dd322197e889 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.826385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 435/856] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:01.831499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.831788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.832066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.839727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:01.844092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:01.850830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:01.852011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:01.901875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d00a0c-4ca0-4f9f-92ba-656547bc51f2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:01.938259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:01.938941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:01.939233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:01.939520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:01.940239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:01.940556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:01.941967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:01.944411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 436/857] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:01.948107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 422/858] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:01.968272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.082536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cc3cf-c706-4725-8262-4e668f5c7ccd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.122305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:02.123115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.123400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.123676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.126465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:02.135870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:02.143715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:02.144731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:02.145782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 437/859] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:15:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:02.151642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.272554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9f5f-3a92-4681-af81-39f7e62556f9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.290909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:02.308357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:02.309080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.309811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.310093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.310858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:02.311171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:02.312672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:02.321876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 423/860] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:15:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:02.327087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.440867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16123085-d4e3-4b3f-a4b2-2e0e5f286794 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.478727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:02.479906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.480393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.480843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.493116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:02.494820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-eb1e11b2-8146-4f9b-a464-05fb87c9f722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:02.514594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:02.515540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:02.516407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:02.516407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:02.516407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:02.516407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:02.518364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 438/861] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:02.528931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.632471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279693-f7d9-4d37-815c-ba01dd85c1aa None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.632834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279693-f7d9-4d37-815c-ba01dd85c1aa None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.633092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279693-f7d9-4d37-815c-ba01dd85c1aa None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.634810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 439/862] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:02.640419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.640722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.641025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.665104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:02.666340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:02.729736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:02.761241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 440/863] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:02.778538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.784728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b043f40-5151-4195-9c7a-1d24a119b347 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.824687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:02.825413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.825700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.825977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.827117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:02.827117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:02.828358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:02.841341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 424/864] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:02.846332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:02.898737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:02.901767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2311c-9b1a-43e4-8fe7-68d25c4637ab None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.956604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:02.957680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:02.959310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:02.959928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:02.960897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:02.962584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:02.973630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c012d-8ef0-47cb-a341-a13ff5d1740f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:02.980147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 441/865] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:02 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:03.026298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:03.027189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.027480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.027760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.037594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:03.039152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:03.056689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:03.057714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:03.058631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 425/866] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:15:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:03.063905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:03.114670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:03.187016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc4790-c5fc-45a9-8a5d-d413477ce9f8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:03.223273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:03.224516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.224808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.232034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.232912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:03.233283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:03.234705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:03.260082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 442/867] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:15:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:03.265389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:03.373537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:03.376719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6ae5a-fa60-4bd1-8372-7b94467e2b0b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:03.412924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:03.413586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.413871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.414149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.484157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:03.485628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:03.590118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:03.599540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 426/868] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:03 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:03.601041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:03.704940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:03.712037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5ab7c-ad6a-4d61-a02d-0c7304853fb9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:03.748216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:03.749029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.749322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.749606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.760339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:03.761842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0e904afb-6225-46a7-8f0e-11b7a2c0acd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:03.781373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:03.782294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:03.783212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:03.783212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:03.783212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:03.783212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:03.783212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 443/869] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:03.789218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:03.903059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5901d7f-e18e-4057-a2e9-1d5cb19c8885 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:03.934352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac23c1f-35df-4885-a989-e7485776989a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.934657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac23c1f-35df-4885-a989-e7485776989a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.934916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac23c1f-35df-4885-a989-e7485776989a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.936669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 444/870] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:03.941466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.941761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.942044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.945085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:03.945945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:03.946379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:03.946813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:03.947683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:03.948142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:03.949674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:03.961773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:03.962885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 427/871] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:03.967274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:03.967925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.029714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.060417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 445/872] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:04.080348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.081023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad9d6-574b-4346-b1a8-c2767b5bc47d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.116517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.117608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.118033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.118610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.125710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.127321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:04.144004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:04.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:04.146681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:04.148951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 428/873] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:04.154750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.176409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.206570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e336d20-c10d-4d35-be67-f2a75bc5fbb4 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.251830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.252665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.252972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.253299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.253980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.256209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:04.269323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:04.270607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:04.271605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:04.276663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 446/874] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:04.277657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.291537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c602471-fed2-4f89-a612-b3862bec1c1e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.312220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.332303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.333774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.334067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.334348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.335250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:04.335676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.337137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78382585-6d68-46de-8bb1-eecf69333c51 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.350533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 429/875] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:04.356724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881aecb-6236-4764-acc7-75146262268e None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.357146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881aecb-6236-4764-acc7-75146262268e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.357427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881aecb-6236-4764-acc7-75146262268e None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.362712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 430/876] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:04.362712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881aecb-6236-4764-acc7-75146262268e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.396691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04df53-b025-49fa-b3b3-01d25e539c43 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.433844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.434523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.434815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.435099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.435811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:04.436131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.437534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969c296-406b-4b61-8e05-c643d88f93eb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.442776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 447/877] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:04.443930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.444287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.444579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.455984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.463319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:04.465431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:04.477169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881aecb-6236-4764-acc7-75146262268e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.513331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.513986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.514275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.514555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.515394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d58b01886c5d441da40130bffc817da7) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:04.515709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.517163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.523291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 431/878] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:04 2022] GET /identity/v3/role_assignments?user.id=d58b01886c5d441da40130bffc817da7&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:04.523883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.568930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 448/879] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:04.586110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:04.604363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.674358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb05f8-1d32-45c5-b43e-2d1441cdc428 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.701457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb5195-a3ea-4aab-9ffd-3bc50d6b836d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:04.720724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.724416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.724416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.724416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.724416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:04.724416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.724845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b20a37-cddb-4e8f-8b6b-6637ef0f5f34 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.737195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 432/880] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:04.756465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:04.757657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:04.757657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:04.757844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:04.758536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:04.759964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd8ed5-cbe3-469a-be40-e9d96ebb91c0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:04.763591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:04.782083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 449/881] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:04 2022] POST /identity/v3/services => generated 269 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:05.225339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6af9f-a9b7-4212-967e-ec75754d0b9b None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:05.225637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6af9f-a9b7-4212-967e-ec75754d0b9b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:05.225914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6af9f-a9b7-4212-967e-ec75754d0b9b None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:05.227548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 433/882] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:05.232907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:05.233213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:05.233494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:05.250833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:05.252007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:05.329496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 450/883] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:05.352292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6af9f-a9b7-4212-967e-ec75754d0b9b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:05.462454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6af9f-a9b7-4212-967e-ec75754d0b9b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:05.465028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:05.501687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:05.502415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:05.502701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:05.502980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:05.503644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:05.505131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:05.517502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:05.518544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:05.519355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 434/884] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:05.525619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:05.627604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7216c6-c677-4808-a8ab-272d01b2df2b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:05.651436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:05.664178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:05.664858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:05.665168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:05.665444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:05.666152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:05.666459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:05.667832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:05.672906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 451/885] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:05.679061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:05.799694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba56d1-b7ed-4288-ae81-d560a7daea5f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:05.839415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:05.840092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:05.840378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:05.840657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:05.841401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:05.841715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:05.843086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:05.849078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 435/886] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:05.854641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:05.977159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e71285-094b-443b-8d90-12b1ace0568c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:06.014494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:06.015162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.015456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.015741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.016547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] RBAC: Adding query filter params (interface=public, service_id=bd205904dc0e462392921fd96b3b5a27, region_id=RegionOne) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:06.016864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:06.018278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef9263-a06e-424f-8555-911fb3ff397f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:06.027003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 452/887] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:15:05 2022] GET /identity/v3/endpoints?service_id=bd205904dc0e462392921fd96b3b5a27&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:06.032593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:06.067440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:06.139603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420c959-3d08-43c5-b3e7-3d660d2ea7cb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:06.174344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:06.175008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.175295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.175574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.176259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:06.177729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5bfcc-99a4-49ec-a2af-8ee5c99fe078 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:06.183749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 436/888] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:06 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:15:06.901005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66f2e-5dd8-4fc0-a7cf-8e62e86e0641 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.901551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66f2e-5dd8-4fc0-a7cf-8e62e86e0641 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.901551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb66f2e-5dd8-4fc0-a7cf-8e62e86e0641 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.903303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 453/889] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:06.909401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.910401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.910968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.929773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:06.931355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:06.971370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1eadb-af8b-4f19-a19d-a44d4b7564eb None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.972024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1eadb-af8b-4f19-a19d-a44d4b7564eb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.972512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1eadb-af8b-4f19-a19d-a44d4b7564eb None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.974543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 454/890] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:06.980001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ccc9c-fb1b-4f80-8174-f857b8fd4b55 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:06.980537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ccc9c-fb1b-4f80-8174-f857b8fd4b55 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:06.981049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ccc9c-fb1b-4f80-8174-f857b8fd4b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:06.999744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7c2ccc9c-fb1b-4f80-8174-f857b8fd4b55 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:07.001830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7c2ccc9c-fb1b-4f80-8174-f857b8fd4b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:07.013030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 437/891] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:07.031256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.057556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.069680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.081392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 455/892] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:07.094108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d416ac-1600-4243-b45d-9d9b55420f45 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.094635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d416ac-1600-4243-b45d-9d9b55420f45 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.095071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d416ac-1600-4243-b45d-9d9b55420f45 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.096794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 456/893] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:07.103042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.103528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.103976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.123861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:07.125095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:07.148056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6037b5e-7ffa-4f25-9130-39f716f451cd None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.184100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.185190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.185480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.185754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.197427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 457/894] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:07.198020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.203299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:07.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:07.212746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:07.213673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:07.221076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 438/895] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:07.222132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.313903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc9db5-b19d-495d-aff9-752f76493000 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.319145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.337559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9d8bd-e70f-4793-8dbc-bd3fcfc711d0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.339442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.367825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.368506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.368807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.369091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.369863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.371200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.375845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68d075-4676-446d-af20-30989270c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.376669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68d075-4676-446d-af20-30989270c284 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.376995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68d075-4676-446d-af20-30989270c284 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.377325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68d075-4676-446d-af20-30989270c284 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.378204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68d075-4676-446d-af20-30989270c284 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.379953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68d075-4676-446d-af20-30989270c284 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.395890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 458/896] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:07 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:07.395890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4a68d075-4676-446d-af20-30989270c284 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:07.396310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:07.397237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:07.398045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 439/897] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:07.404411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68d075-4676-446d-af20-30989270c284 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.530991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d571763-dd43-475e-93aa-144a6008d4f4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.545480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68d075-4676-446d-af20-30989270c284 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.575301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.575826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.576119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.576404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.577243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:07.577520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.578895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.588478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 459/898] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:07.591213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.591890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.592173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.592450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.593640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:07.593640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.594903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.595152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.600113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 440/899] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:07.605849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.624527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.630993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.714237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74548d5-6620-4b6b-8a2a-a59b8387f65a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.730805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987d839-dc82-4404-bf5a-8cf3fee3e475 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.756038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.757159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.757589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.758040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.768523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.769236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.769525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.769808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.769971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.771225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:07.771635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.772525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.773050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.779219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 441/900] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:07.784501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:07.791801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:07.792860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:07.794040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:07.794040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:07.794040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:07.794040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:07.795945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 460/901] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:07.802061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.871125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.886195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:07.897064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d882537-6895-4677-9c81-b4813845ae71 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.906929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f582e8-4e4f-4053-85cb-7b055970d9cd None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:07.934290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.934959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.935245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.935756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.936412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] RBAC: Adding query filter params (interface=public, service_id=87e245f7378248979994387617f7ea60, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:07.936791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.938148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.944895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:07.945810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:07.946297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:07.946754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:07.947663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:07.948162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:07.949114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 442/902] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/endpoints?service_id=87e245f7378248979994387617f7ea60&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:07.950011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:07.950454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:07.968937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 461/903] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:07.970099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.064472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acb624-e4a4-44fc-a40d-4f87f5d0b167 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.079095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103f4fb-e73b-454a-b4aa-6891bac13d4b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.099500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.100339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.100694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.101095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.101866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.103309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.110081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 443/904] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:15:08.121234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.122016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.122306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.122592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.131554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.133090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.136397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:08.151266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:08.152304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:08.153280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 462/905] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:15:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:08.160253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.171209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.277930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03255264-0ea3-4d99-bc92-9f4ad1245546 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.322304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.323000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.323293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.323580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.324506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:08.324828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.326355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de7030-8f41-4152-ad63-1f7da256cd17 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.348275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 444/906] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:08.354982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.405992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886189-0f77-4196-a6cd-587bd3231418 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.406287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886189-0f77-4196-a6cd-587bd3231418 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.406490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886189-0f77-4196-a6cd-587bd3231418 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.408843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 445/907] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:08.414595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.415040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.415462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.436051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:08.437438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:08.475600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04b9d6-4991-4f2b-8633-6fd472c6ba99 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.512659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.514538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 446/908] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:08.523648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.524328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.525269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.525796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.538241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.604846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.607589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.609634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.641858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dcace-aee1-42e7-adcf-512445dc769e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.680651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.681429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.681712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.681989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.682659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.684028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:08.698552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:08.699600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:08.700417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 447/909] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:08.705612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.735674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 463/910] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:08.742108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.789144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.834721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be03b9-0efc-47cf-803b-54da5af70867 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.845969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:08.849659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45aef-74fe-4036-95c7-0a01dc45da28 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:08.873028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.874114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.874595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.875140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.876077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:08.876589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.878361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.884393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 448/911] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:08.892487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:08.892687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:08.892771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:08.892771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:08.892771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:08.903410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:08.904948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:08.924265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:08.925798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:08.927469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:08.927469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:08.927469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:08.927469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:08.927469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 464/912] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:08.933880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.009562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af69b9-055c-4aa2-ad5d-c8f5ba3ae21e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.034322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:09.047397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.048385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.049005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.049512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.050502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:09.051006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.053656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.059205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 449/913] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:09.066015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.075318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7577b-dba1-4894-9053-5b46973ad576 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.110652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.111691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.112172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.112663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.113607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:09.114658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.116323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.129105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 465/914] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:09.135627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.157468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:09.190217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79c6d5-8995-4216-b769-5dda447446dd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.227236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.227955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.228258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.228965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.229558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.230929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.264933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 450/915] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:09.277843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9e505-70b2-46df-8e8b-bb7efb238e5f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.329832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.330655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.330936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.331208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.339486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.341100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:09.357776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:09.358723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:09.359597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 466/916] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:09.370078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.420783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:09.488753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c578eb-ed57-4989-a2bf-9da096eb0a6e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.526401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.527441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.527935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.528418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.529467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:09.530116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.531798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89afac-4409-489f-baf6-a6965bc85cde None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.545701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 451/917] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:09.552084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.627600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:09.668939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeeded3-04a1-49fc-a5ef-5c231ae6fade None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.710455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:09.711151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.711436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.711717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.712572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] RBAC: Adding query filter params (role.id=13eec5c3ca2440f6a6e219d4a1da1dd1, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d58b01886c5d441da40130bffc817da7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:09.712903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:09.714381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0acb6-c6a5-4748-9d7a-ebf40488a00c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:09.721650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 467/918] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:09 2022] GET /identity/v3/role_assignments?user.id=d58b01886c5d441da40130bffc817da7&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:09.755495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e7d5-91bc-4dc9-b7c2-ae7b15b5095f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.755781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e7d5-91bc-4dc9-b7c2-ae7b15b5095f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.755973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953e7d5-91bc-4dc9-b7c2-ae7b15b5095f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.757707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 452/919] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:09.763397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:09.763699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:09.763972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:09.783972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:09.785217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:09.862458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 468/920] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:09.885678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953e7d5-91bc-4dc9-b7c2-ae7b15b5095f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:09.995233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953e7d5-91bc-4dc9-b7c2-ae7b15b5095f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:09.997167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.037099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.037890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.038233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.038525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.039194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.040568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:10.054069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:10.055035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:10.055856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 453/921] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:10.063145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.131475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.172082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697f92-ea08-4c4e-a0fe-712484ea705b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.207766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.208591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.208894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.209175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.209892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:10.210200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.211565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1bb00-aa66-45da-8b8c-4995c79d9583 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.216912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 469/922] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:10.222561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.335098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f262c-4380-4f98-aac7-5bfe3ff94e51 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.371006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.371691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.371984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.372268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.373023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:10.373342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.374110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674830-83d9-4946-9a1b-6cdc6007c551 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.374408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674830-83d9-4946-9a1b-6cdc6007c551 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.374689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37674830-83d9-4946-9a1b-6cdc6007c551 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.374793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.380369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 470/923] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:10.380642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 454/924] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:10.382075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.382371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.382656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.387133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.404153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:10.405671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:10.420318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.505107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 471/925] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:10.507552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3caf2-7215-48d2-8e16-b95fb72be89b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.530006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.552118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.553595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.554249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.554808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.556552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] RBAC: Adding query filter params (interface=public, service_id=2641032c0a094a31bca22b9f61f51798, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:10.557165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.559306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.568853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 455/926] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/endpoints?service_id=2641032c0a094a31bca22b9f61f51798&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:10.574606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.599856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.654000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc8a3f-3345-4321-b7f2-ed713c9617c7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.672084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.678907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751d703-dd6c-441d-9a5b-c31d471e60d6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.694460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.695651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.696203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.696688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.697579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.699272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:10.713452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:10.714388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 00:15:10.715202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:10.717933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 472/927] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:10.719745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.720569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.721516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.721950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:10.727077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.728630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.740939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 456/928] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:15:10.850193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14a9a-7d37-4466-989a-4a82dba1c566 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:10.875620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:10.892524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:10.893529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:10.893988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:10.894427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:10.895353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:10.895867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:10.897466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:10.903640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 473/929] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:10.909030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:11.020786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc755f4f-9675-430f-afb3-ae99922d86af None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:11.054951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:11.055693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.055897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.056174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.056965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:11.057278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:11.058688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9046214-3c47-4537-9c62-2869ea8e63e6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:11.064027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 457/930] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:11.070651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:11.172094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f184-8679-44de-b5e5-1b5ecf39c60b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:11.206104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:11.206772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.207060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.207342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.208826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:11.210217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cfd13-dc2f-431f-9561-aff0e834ccab None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:11.230613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 474/931] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:11.566037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f6aa3-fb9c-45bf-804e-322b400872c2 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.566396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f6aa3-fb9c-45bf-804e-322b400872c2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.566601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f6aa3-fb9c-45bf-804e-322b400872c2 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.568768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 458/932] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:11.574038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.574329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.574609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.592635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:11.593937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:11.630543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:11.670913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 475/933] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:11.694208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f6aa3-fb9c-45bf-804e-322b400872c2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:11.729354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:11.810699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f6aa3-fb9c-45bf-804e-322b400872c2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:11.844632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:11.845244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.845527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.845799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.846513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:11.847932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:11.849867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING keystone.server.flask.common [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 00:15:11.869058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.sql.core [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:15:11.869058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 00:15:11.869058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [parameters: {'id': '1386b9b950d74256811034c3d540efde', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 00:15:11.869058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86548) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-e91fdca9-25b1-4789-8e6b-b657a3e1b141 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1386b9b950d74256811034c3d540efde.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1386b9b950d74256811034c3d540efde. Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:15:11.886615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application result.read() Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 00:15:11.887508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application yield resource Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise self.value Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application session.commit() Jun 16 00:15:11.888952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:15:11.891174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:15:11.897290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application result.read() Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:15:11.899211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [parameters: {'id': '1386b9b950d74256811034c3d540efde', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:11.901721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:11.903611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1386b9b950d74256811034c3d540efde. Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:11.905788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 459/934] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Jun 16 00:15:11 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:11.907074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:11.967720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb479-4aa9-44b2-b382-527e8a3c4487 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.968090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb479-4aa9-44b2-b382-527e8a3c4487 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.968427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fb479-4aa9-44b2-b382-527e8a3c4487 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.978357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 460/935] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:11 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:11.979823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:11.980172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:11.980504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:11.999109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:12.000607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:12.010401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7175bba-4a24-466a-b42f-3c0107901d50 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.048097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.048931 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.049227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.049512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.057480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:12.059172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:12.074916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:12.076465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:12.077556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:12.079658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 476/937] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:12.084861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 461/937] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:12.086012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.097084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:12.104022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.133915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:12.200630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf138-53d7-4537-84d2-39328b5cdec1 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.213093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2369d9-2d8b-4768-9074-27cea311f0ef None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.237995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.238526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.238914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.239247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.240141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:12.240517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:12.242161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:12.250498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.251531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.252002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.252480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.268915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 462/938] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:12.273503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:12.283800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:12.284808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:12.286678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 477/939] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:12.292314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.373892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:12.415166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b91c8-27ba-4173-8a6c-aafa60113c2f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.456524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.459909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.460450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.460962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.461935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:12.462455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:12.464044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:12.474838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 463/940] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:12.482095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.583030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:12.600392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7fca9-4360-4837-8ad4-9321887c7897 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.636011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.636733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.637054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.637338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.656202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:12.657997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:12.678492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:12.679421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:12.680279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:12.680279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:12.680279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:12.680279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:12.680279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 478/941] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:12.686128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.797868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a321d-efc7-4bbf-ba34-0be24975fb6c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:12.828579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:12.835554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:12.836233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:12.836526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:12.836872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:12.837646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:12.838009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:12.839389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:12.852351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 464/942] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:12.857978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:12.967211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd255442-e1e3-4576-bff4-d77d6d83431f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.008399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.009748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.010235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.010729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.020767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.022711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.025331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-9ba6eafb-ed45-4121-89d7-e94d8fa76014 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:13.040413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:13.041450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:13.042369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.044990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 479/943] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:15:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:13.050448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.073142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296da8-0ddb-4c2b-8634-c8c185fe4662 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.073770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296da8-0ddb-4c2b-8634-c8c185fe4662 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.074363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296da8-0ddb-4c2b-8634-c8c185fe4662 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.076256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 480/944] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:13.081119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649906-c822-4239-a637-0eb5114f505b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.081668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649906-c822-4239-a637-0eb5114f505b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.082141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649906-c822-4239-a637-0eb5114f505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.101046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e8649906-c822-4239-a637-0eb5114f505b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:13.102587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e8649906-c822-4239-a637-0eb5114f505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:13.159430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f747f-442e-47a1-9de6-ce78694fa367 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.173817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 481/945] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:13.197013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8649906-c822-4239-a637-0eb5114f505b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.197362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a74c-310b-4297-81d6-420953121bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.197967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a74c-310b-4297-81d6-420953121bac None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.198259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a74c-310b-4297-81d6-420953121bac None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.198542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a74c-310b-4297-81d6-420953121bac None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.199402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044a74c-310b-4297-81d6-420953121bac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:13.199717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044a74c-310b-4297-81d6-420953121bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.201343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044a74c-310b-4297-81d6-420953121bac None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.221415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 465/946] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:13.227647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a74c-310b-4297-81d6-420953121bac None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.283276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.320393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8649906-c822-4239-a637-0eb5114f505b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.344594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a74c-310b-4297-81d6-420953121bac None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.349708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.358142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.359464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.359848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.360225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.361145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.362591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:13.375492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:13.376743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:13.377815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.381047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 482/947] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:13.386702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.398224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.398855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.399144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.399423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.411948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.413500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:13.432910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:13.433887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:13.434808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:13.434808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:13.434808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:13.434808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:13.438837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 466/948] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:13.440103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.505010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e715153-c8b4-40fd-9efa-bacbbf787b2b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.543295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.545500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.546193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.546482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daadc31-157b-42fd-b5b7-abff45bc6904 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.546587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.546775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.547479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:13.547794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.549189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04c776-c588-4ee0-a594-75b2970b480e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.555868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 483/949] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:13.557321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e393ab0-8f3c-469b-a61a-efc982336485 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.557618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e393ab0-8f3c-469b-a61a-efc982336485 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.557898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e393ab0-8f3c-469b-a61a-efc982336485 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.559291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 484/950] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:13.564376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e393ab0-8f3c-469b-a61a-efc982336485 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.588813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.589812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.590106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.590386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.591125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:13.591440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.592815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a33caa-9245-4482-be9a-e4f8591164ca None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.609096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 467/951] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:13.609096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.609096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.609096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.623798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:13.625050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:13.629141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.679107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e393ab0-8f3c-469b-a61a-efc982336485 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.710518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 468/952] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:13.711602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.717704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.718389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.719004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.719004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.719751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:13.720070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.721995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.729623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 485/953] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:13.730957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.823260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.834133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f886e2b-591f-46d7-9b42-ad56a35b7be0 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.837135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56f471-4ec0-4722-80a1-60ab2a8b44f6 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:13.856680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:13.872751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.872972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:13.873625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.873914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.874006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:13.874191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] PATH_INFO: `/v3/services/compute` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.874498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:13.874843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.874934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:13.876207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.882845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:13.884424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:13.887817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:13.889112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:13.890019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 486/954] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:13.890847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:13.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:13.907850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:13.908836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 469/955] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:13.910216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:14.007317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5d41-ab6f-42c8-b961-cbec927b38bb None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.015784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c06d-8b97-4dfb-a90e-44960d13e634 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.044220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.044908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.045191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.045515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.046199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.047557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.053030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.053712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.054001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.054285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.055007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] RBAC: Adding query filter params (name=compute) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:14.055320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.057010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.062582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 470/956] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:14.063550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:14.071796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 487/957] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:14.072954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:14.086738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:14.138490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:14.197083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdb545-57ed-4117-a537-62415257cd0d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.220905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a311eab-65af-43fd-919e-d1e1371e4367 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.240733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.241813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.242276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.242710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.243695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:14.244181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.246057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.261089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.261910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.262298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.262677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.263338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 471/958] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:14.263584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] RBAC: Adding query filter params (type=compute) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:14.263981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.265508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b1e8d-5c7d-4539-a3e4-b7b37459ceee None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.268968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:14.272696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 488/959] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:14.369327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:14.397737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92907b0-b126-43be-90cc-eca521f9f9d0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.437777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.438685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.438968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.439246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d58b01886c5d441da40130bffc817da7/roles/13eec5c3ca2440f6a6e219d4a1da1dd1` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.441497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=d58b01886c5d441da40130bffc817da7, role_id=13eec5c3ca2440f6a6e219d4a1da1dd1)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.443630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea683467-77f5-4fed-a507-07da1cd79154 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.459706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 472/960] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:15:14 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d58b01886c5d441da40130bffc817da7/roles/13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:15:14.598249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde368b-1ad3-4b9a-baac-1cb2c32d9aad None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.598901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde368b-1ad3-4b9a-baac-1cb2c32d9aad None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.599312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde368b-1ad3-4b9a-baac-1cb2c32d9aad None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.601257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 489/961] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:14.606544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.606836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.607119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.624976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:14.626104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:14.711332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 473/962] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:14.744000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde368b-1ad3-4b9a-baac-1cb2c32d9aad None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:14.848281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:14.853228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde368b-1ad3-4b9a-baac-1cb2c32d9aad None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:14.889170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:14.890201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:14.890687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:14.891198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:14.892146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:14.893809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:14.896143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING keystone.server.flask.common [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 00:15:14.910208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.sql.core [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:15:14.910208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 00:15:14.910208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [parameters: {'user_id': '2ac47707eed34b50a4ec880da7f8e690', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 00:15:14.910208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86549) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result.read() Jun 16 00:15:14.917800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:15:14.918639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application yield resource Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application next(self.gen) Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise self.value Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application session.commit() Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 00:15:14.919575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.session.flush() Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application compat.raise_( Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 00:15:14.920455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 00:15:14.921358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application conn.query(q) Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application result.read() Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 00:15:14.922223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [parameters: {'user_id': '2ac47707eed34b50a4ec880da7f8e690', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:14.923032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 00:15:14.923801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:14.924684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 490/963] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Jun 16 00:15:14 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:14.930275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:15.021045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:15.044487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1d808-d977-498f-85f1-7e1b47484bc7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:15.081819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:15.083610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:15.084109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:15.084580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:15.092426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:15.094169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-794e6ece-6ec2-4242-8577-5d80a1dd0f9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:15.109745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:15.110719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 00:15:15.111649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:15.114991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 474/964] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:15.121247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:15.231491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a3d5-c537-4ee9-bf41-574421e3428d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:15.269253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:15.270716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:15.271162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:15.271588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:15.272543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:15.273027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:15.274830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41c878-db4a-428a-8c67-c0eb1acf182f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:15.315556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 491/965] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Jun 16 00:15:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:16.372167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26ed6-1f42-4ecc-8e15-50c19173a794 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.372687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26ed6-1f42-4ecc-8e15-50c19173a794 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.372896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26ed6-1f42-4ecc-8e15-50c19173a794 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.374146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 475/966] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:16.379878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70b972-d8a9-44f1-b34b-675b36469c17 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.380258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70b972-d8a9-44f1-b34b-675b36469c17 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.380540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70b972-d8a9-44f1-b34b-675b36469c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.397788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-ee70b972-d8a9-44f1-b34b-675b36469c17 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:16.399289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ee70b972-d8a9-44f1-b34b-675b36469c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:16.415397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:16.477070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 492/967] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:16.500286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26ed6-1f42-4ecc-8e15-50c19173a794 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:16.521629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b547-e56d-4399-9815-d7d45c6dfe22 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.521962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b547-e56d-4399-9815-d7d45c6dfe22 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.522199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85b547-e56d-4399-9815-d7d45c6dfe22 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.523575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 493/968] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:16.529062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.529355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.529635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.546272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:16.547448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:16.606161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26ed6-1f42-4ecc-8e15-50c19173a794 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:16.619988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 494/969] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:16.633910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:16.642997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:16.647870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:16.648695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.649019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.649299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.650229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:16.651609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:16.664174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:16.665239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:16.666077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 476/970] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:16.671244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:16.729843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:16.751286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fcd44-5f37-4c91-8fa7-5ca0f329619e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:16.772084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29c10-f03d-4347-a195-088705f1ef32 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:16.785357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:16.786126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.786414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.786691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.787411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:16.788785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e51425-bc9c-4abc-bcf7-9546afad3d1f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:16.803513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 495/971] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:16 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:16.804787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecb66-aa56-4a0b-b896-89591391f990 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.805113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecb66-aa56-4a0b-b896-89591391f990 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.805399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ecb66-aa56-4a0b-b896-89591391f990 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.806772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 496/972] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:16.812490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.812785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.813100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.817762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:16.819101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.819459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.819809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.820625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:16.821060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:16.822555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:16.828177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 477/973] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:16.834221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:16.837516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:16.838752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:16.889866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:16.916532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 497/974] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:16.940690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:16.948735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755ca7-4526-4e6a-8505-b9638525df72 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:16.989768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:16.990443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:16.990727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:16.991006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:16.991707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:16.992020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:16.993417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:16.999621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 478/975] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:17.006116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.021644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.058180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6915138-b3a2-4551-a2ca-430b4e31b5c8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.093866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.094718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.095007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.095405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.106798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:17.112109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30bf6e-18d4-4f8a-a713-f8e8250e846b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:17.116249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:17.117421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:17.118249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:17.122343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 498/976] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:17.123095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.127556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.150296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.150864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.151150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.151430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.152231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] RBAC: Adding query filter params (interface=admin, service_id=2641032c0a094a31bca22b9f61f51798, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:17.152547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.153942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.159290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 479/977] 10.222.0.23 () {56 vars in 1249 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/endpoints?service_id=2641032c0a094a31bca22b9f61f51798&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:17.165185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.230533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a8ae1-6d73-4f27-994a-ac5c8b75c45b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.259865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.266927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040e5be-ca25-492b-a145-f47a6ed21d47 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.268473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.270474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.270766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.271101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.271836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:17.272149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.273554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.283104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 499/978] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:17.289126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.307530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.308210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.308503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.308788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.309520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.310898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.317508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 480/979] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:15:17.396921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88c1f7-e8f8-4052-b587-6fe1db625af1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.433689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.434415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.434700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.434974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.446805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.448289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:17.467272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:17.468346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:17.474488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:17.474488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:17.474488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:17.474488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:17.474488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 500/980] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:17.477442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.525784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.585459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397af861-3a4f-4420-af24-3fcef6f0656e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.619899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.621140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.621593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.621876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.622628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:17.622946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.624901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.642950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 481/981] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:17.644623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.693486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.760941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35edc55-a037-480c-bbf5-df747f638790 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.794563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.795375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.795658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.795936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.803627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.805093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:17.821652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:17.822641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:17.823606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 501/982] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:17.827039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:17.927257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:17.930329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557757d-ffbf-4cca-b09c-03f2c852de00 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:17.968687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:17.969320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:17.969604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:17.969881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:17.970728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:17.971040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:17.972469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:18.002623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 482/983] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Jun 16 00:15:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:18.004005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:18.115758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd369-6951-4e15-9ad0-2ebc4141d8d9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:18.118140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:18.152514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:18.153273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:18.153565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:18.153847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:18.221515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:18.222970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:18.324443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 502/984] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:18 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:18.325471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:18.442616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2f5bc-eaa4-4cb1-9aba-72b2734300e3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:18.478239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:18.478990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:18.479253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:18.479781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:18.490926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:18.492385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:18.511229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:18.512156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:18.517903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:18.517903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:18.517903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:18.517903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:18.517903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 483/985] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:18.518548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:18.521784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:18.621214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227b57-cabc-431a-bfee-49d6b5e5002c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:18.655430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:18.656106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:18.656398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:18.656682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:18.657437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:18.657755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:18.659118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:18.675687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 503/986] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:18.677738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:18.690116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:18.806298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4ce11-62e4-4d85-bb34-95e57c696013 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:18.840300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:18.841122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:18.841410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:18.841687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:18.848330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:18.849797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8c5ac28a-a13f-4e85-a3e5-4a4684aab8b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:18.864033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:18.865185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:18.866078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 484/987] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:18.872355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:18.937620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:18.979140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ef7f-bbfe-492b-8ad4-3cf8c6f4fdf4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:18.998378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efe159-9238-44c2-8dfe-ee3f8e9493f9 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:18.998692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efe159-9238-44c2-8dfe-ee3f8e9493f9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:18.998945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efe159-9238-44c2-8dfe-ee3f8e9493f9 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.000449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 485/988] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:19.005982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.006251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.006536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.015024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.015912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.016373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.016823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.017839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.018153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.019577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.023448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:19.024654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:19.032349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 504/989] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:19.039087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.097263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.102965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 486/990] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:19.125699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.150312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b8e591-e1db-4eb9-82ce-b2cd98c11b2a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.176568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.188477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.189222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.189515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.189802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.190785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] RBAC: Adding query filter params (role.id=13eec5c3ca2440f6a6e219d4a1da1dd1, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d58b01886c5d441da40130bffc817da7) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.191106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.193538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.200549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 505/991] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/role_assignments?user.id=d58b01886c5d441da40130bffc817da7&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:19.243546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7a269-a41c-4f58-b7c9-1dbc7daf415b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.281760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.282582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.282871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.283180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.283871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.285901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:19.298719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:19.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:19.300625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:19.306349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.306802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 487/992] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:19.418423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2d819-796e-4821-a238-baf1f0dd9f68 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.456151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.456827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.457147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.457945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.458668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.458982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.460354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d72e77-7543-47d1-acfb-c3e8260a5bf0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.465554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 506/993] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:19.471355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.487748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef497-83d1-4a3b-b6b9-7a806b4cec25 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.488067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef497-83d1-4a3b-b6b9-7a806b4cec25 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.488326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afef497-83d1-4a3b-b6b9-7a806b4cec25 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.490812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 507/994] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:19.495665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.495955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.496237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.519338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:19.521402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:19.540764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.573229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.584010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbd842-c6f5-4ad6-9cde-6fae36ecd593 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.601049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 508/995] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:19.619964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.626111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.626857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.627152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.627432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.628668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.629063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.631912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.639424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 488/996] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:19.644700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.731648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf2d2-56ef-405e-8950-f5293ef771df None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.763012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96546a07-e206-4956-8f38-dc4b2d8ac877 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.766651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.767471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.767760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.768041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.768705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.770639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:19.783259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:19.784251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:19.785310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 509/997] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:19.790823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.794420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.803691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.804535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.804755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.805077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.805890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] RBAC: Adding query filter params (interface=public, service_id=a944dbe642bb4acd9291b6fb329d3975, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.806212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.807587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.812980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 489/998] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/endpoints?service_id=a944dbe642bb4acd9291b6fb329d3975&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:19.818952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.842099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:19.896782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ef634-2a91-4328-8422-6bb2eab2ad7f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.929095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc89a5-90fe-4e70-8848-755a5e53f637 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:19.931401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.932072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.932360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.932635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.933372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:19.933682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.935043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.940722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 510/999] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:19.949126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:19.970136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:19.970793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:19.971074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:19.971349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:19.972036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:19.973585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:19.989007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 490/1000] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:15:20.064138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec294ed8-edd4-4a5f-b25f-d0be90ec6d67 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:20.078973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:20.108535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:20.109292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:20.109581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:20.109864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:20.110653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:20.110972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:20.112375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02753279-0b62-4ee0-bc19-a3232b713a5e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:20.123232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 511/1001] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Jun 16 00:15:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:20.124907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:20.189416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:20.235051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48062a-2bb1-480c-bea9-690a5da89d80 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:20.270398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:20.270977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:20.271261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:20.271537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:20.272291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:20.273673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1f62b-6998-4afc-8bd7-380323ecf41c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:20.290139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 491/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:21.406409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01bf12-8120-4790-8667-40b1607ea06f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:21.406934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01bf12-8120-4790-8667-40b1607ea06f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:21.407044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01bf12-8120-4790-8667-40b1607ea06f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:21.408513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 512/1003] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:21.413952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:21.414349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:21.414510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:21.431296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:21.432495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:21.525224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 492/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:21.543575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01bf12-8120-4790-8667-40b1607ea06f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:21.646302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01bf12-8120-4790-8667-40b1607ea06f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:21.679801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:21.680518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:21.680807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:21.681112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] PATH_INFO: `/v3/services/image` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:21.681770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:21.683128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:21.695182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:21.696187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:21.697093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 513/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Jun 16 00:15:21 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:21.703794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:21.753082 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:21.814229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185fe22-fd9c-4bc8-b92d-a7cb7eb917d5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:21.850075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:21.850746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:21.851029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:21.851301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:21.852003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] RBAC: Adding query filter params (name=image) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:21.852312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:21.853702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2b3a9-6e70-4946-b752-d714c6b0ab6c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:21.858738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 493/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:21.864163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:21.905546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:21.968078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48299-128d-40ca-b8cb-cb0f60b9b5e2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.002603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.003279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.003571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.003855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.004636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] RBAC: Adding query filter params (type=image) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:22.004979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.006355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f5754-f929-488c-87b5-292e7a084efe None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.016929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 514/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:22.172503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623244-8b53-4053-8086-6e8101b96250 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.172802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623244-8b53-4053-8086-6e8101b96250 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.173110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623244-8b53-4053-8086-6e8101b96250 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.174492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 494/1008] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:22.179959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c92d8-a1f9-4ff5-9bbe-9f3c467e22fa None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.180253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c92d8-a1f9-4ff5-9bbe-9f3c467e22fa None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.180541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c92d8-a1f9-4ff5-9bbe-9f3c467e22fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.198309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-947c92d8-a1f9-4ff5-9bbe-9f3c467e22fa None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:22.199515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-947c92d8-a1f9-4ff5-9bbe-9f3c467e22fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:22.273009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 515/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:22.293164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c623244-8b53-4053-8086-6e8101b96250 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.406422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c623244-8b53-4053-8086-6e8101b96250 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.415569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:22.444437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26ba29-8e97-4ca4-8e82-f36e3ee7d9cd None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.444975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26ba29-8e97-4ca4-8e82-f36e3ee7d9cd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.445103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26ba29-8e97-4ca4-8e82-f36e3ee7d9cd None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.446523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 516/1010] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:22.446630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.447431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.447725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.448007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.448676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.450152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.451163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.451456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.451742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:22.464834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:22.465948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 00:15:22.466805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:22.469151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:22.469376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 495/1011] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:22.470207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:22.472413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.501507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:22.547204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 517/1012] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:22.571111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.585781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a064b7-cc33-4f2b-b5a7-7e752e879247 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.621506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.622168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.622450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.622725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.623423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:22.623737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.625109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.635507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 496/1013] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:22.636426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.674697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:22.677723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32c294-aed0-4601-bc6d-11ed384c4f45 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.716872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.717723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.718013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.718314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.719488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.720952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.726552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 518/1014] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:22.731827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.755652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84210c1b-d4dc-41f3-9e79-c3f177a75d13 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.763268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:22.793344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.794018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.794329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.794609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.795322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:22.795637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.797008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.803304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 497/1015] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Jun 16 00:15:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:22.809962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:22.842001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af406e2-64c1-438c-aabe-0ed1e7565516 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.881051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.881718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.882004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.882284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.882994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.884415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c165f0e-8959-4d18-acee-cfe66ae1b6f8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.917615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:22.921099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 519/1016] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:22 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:22.922426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed2fca-4d35-4213-bdae-1876a5f95762 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:22.962219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:22.962908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:22.963187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:22.963463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:22.964377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:22.965989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7fcf5-b284-416b-a53a-56b5d6a63f84 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:22.991653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 498/1017] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:24.188769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164497-2572-41c4-bf2d-caa3cc674fae None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:24.189385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164497-2572-41c4-bf2d-caa3cc674fae None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:24.189385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23164497-2572-41c4-bf2d-caa3cc674fae None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:24.190878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 520/1018] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:24.196496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da63d0-a97c-40c0-939b-aef99ce04a0c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:24.196929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da63d0-a97c-40c0-939b-aef99ce04a0c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:24.197130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da63d0-a97c-40c0-939b-aef99ce04a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:24.215129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e4da63d0-a97c-40c0-939b-aef99ce04a0c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:24.216309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e4da63d0-a97c-40c0-939b-aef99ce04a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:24.289866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 499/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:24 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:24.312646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23164497-2572-41c4-bf2d-caa3cc674fae None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:24.395711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:24.416773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23164497-2572-41c4-bf2d-caa3cc674fae None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:24.449643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:24.450242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:24.450529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:24.450811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:24.451548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:24.452908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373eea5-4508-4f8c-b136-40adeda75242 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:24.467579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 521/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:24 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:25.101345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ea296-e0da-4b99-a9de-586c8cf42293 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.101742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ea296-e0da-4b99-a9de-586c8cf42293 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.101918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ea296-e0da-4b99-a9de-586c8cf42293 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.103462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 500/1021] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:25.108527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42956c1-4b97-4c75-81ed-5ab32824b9e1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.108828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42956c1-4b97-4c75-81ed-5ab32824b9e1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.109140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42956c1-4b97-4c75-81ed-5ab32824b9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.126544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c42956c1-4b97-4c75-81ed-5ab32824b9e1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:25.127736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c42956c1-4b97-4c75-81ed-5ab32824b9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:25.208327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 522/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:25.225073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd871-63d2-455b-95c0-4c397ab2e089 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.225544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd871-63d2-455b-95c0-4c397ab2e089 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.225623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbd871-63d2-455b-95c0-4c397ab2e089 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.227127 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 501/1023] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:25.232002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc227-f22f-4692-827d-808faec08287 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.232295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc227-f22f-4692-827d-808faec08287 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.232577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc227-f22f-4692-827d-808faec08287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.233077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbd871-63d2-455b-95c0-4c397ab2e089 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.249628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-921cc227-f22f-4692-827d-808faec08287 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:25.250799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-921cc227-f22f-4692-827d-808faec08287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:25.315406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:25.322438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:25.325013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 523/1024] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:25.346588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cc227-f22f-4692-827d-808faec08287 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.347177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbd871-63d2-455b-95c0-4c397ab2e089 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:25.389279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:25.389998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.390295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.390575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.391761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:25.393231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:25.395081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 502/1025] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:25.401041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.446700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cc227-f22f-4692-827d-808faec08287 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:25.485553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:25.486352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.486643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.486939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.497660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:25.505989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:25.506940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:25.507988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:25.509129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cae461-ed0f-47ed-8801-9dc595deeb74 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:25.509218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 524/1026] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:25.513806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.545100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:25.545774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.546057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.546340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.547047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:25.548471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:25.584922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 503/1027] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:15:25 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:25.596782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:25.647592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081c959-3126-4d55-9ea5-5e1dd4f1ec67 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:25.684499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:25.685215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.685543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.685813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.686578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:25.686890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:25.688277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:25.701074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 525/1028] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:25.705032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.714979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:25.823925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe3cb-62bc-4137-889f-f96811325768 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:25.858886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:25.859687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:25.860031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:25.860310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:25.871221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:25.872651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:25.890055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:25.890943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:25.891816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:25.891816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:25.891816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:25.891816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:25.891816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 504/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:25.899482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:25.999973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546368-c791-4d16-9dea-0597939fe418 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.020482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:26.037110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:26.037784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.038070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.038349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.039143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:26.039454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:26.040891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:26.059098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 526/1030] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:26.060684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:26.124044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:26.175114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c868d-4c17-4d93-a0e5-087a455f5f59 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.215174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:26.215989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.216277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.216557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.226508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:26.228026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:26.245595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:26.246581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:26.247535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:26.249217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 505/1031] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:26.253759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:26.378683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a4004-c087-41a6-b5aa-563a2557db2c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.418872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:26.419720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.420247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.420744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.421790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:26.422376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:26.424461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343435e3-215f-4e12-a8d9-be4ff1a18dbc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:26.450868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 527/1032] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:26.452376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:26.579211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfbdf4-ab67-43dc-8cc3-6185d9bbe626 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.602485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:26.626873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:26.627642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.627932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.628214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.662657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce8608-ce15-479c-9536-7677b7a0959a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.663023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce8608-ce15-479c-9536-7677b7a0959a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.663226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce8608-ce15-479c-9536-7677b7a0959a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.665580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 528/1033] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:26.669548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.669833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.670110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.688114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:26.689339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:26.723838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:26.732825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:26.734878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:26.775289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 529/1034] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:26.800276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:26.864257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 506/1035] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:26.869703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:26.888176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:26.924027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcc83e-cc91-4b8b-86eb-c94970feb852 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.970143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:26.970811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:26.971061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:26.971334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] PATH_INFO: `/v3/services/image` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:26.971997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:26.974637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:26.985523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea7505-a40f-4ed5-afc4-dcb04e16c239 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:26.990811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:26.991798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:26.992640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:26.997707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 530/1036] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:26.999471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.022848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.023653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.023943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.024238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.036502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.037536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.037982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:27.061589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:27.062696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:27.063611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:27.063611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:27.063611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:27.063611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:27.063611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 507/1037] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:27.068061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.120445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b2b6a-5f4b-4c92-b9b1-d208fe18f308 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.138058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.158081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.158754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.159037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.159314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.160653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] RBAC: Adding query filter params (name=image) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.161028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.162396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.168089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 531/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:27.173566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.177848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15973-91e9-460f-bc99-2f38c1c473c0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.212240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.212922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.213212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.213488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.214202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.214512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.215882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.227443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 508/1039] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:27.233164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.279216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.286405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7115e4-b3e6-441e-b4b3-ad5212a72188 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.325033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b103-40b3-440c-891b-067144c7727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.325711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b103-40b3-440c-891b-067144c7727d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.325996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b103-40b3-440c-891b-067144c7727d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.326571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b103-40b3-440c-891b-067144c7727d None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.326983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822b103-40b3-440c-891b-067144c7727d None admin] RBAC: Adding query filter params (type=image) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.327309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822b103-40b3-440c-891b-067144c7727d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.328669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822b103-40b3-440c-891b-067144c7727d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.339114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 532/1040] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:27.340377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b103-40b3-440c-891b-067144c7727d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.343487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061866e4-2a88-4f38-abb2-fb0bdcc8fcd4 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.373524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.383321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.384136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.384472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.384693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.392130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.393601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:27.408731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:27.409697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:27.411994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 509/1041] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:27.416393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.451855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b103-40b3-440c-891b-067144c7727d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.488436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.489159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.489444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.489727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.490546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] RBAC: Adding query filter params (interface=public, service_id=be2d9fa3e39045ceba29e7156a8868fc, region_id=RegionOne) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.490855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.492231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.502438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 533/1042] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/endpoints?service_id=be2d9fa3e39045ceba29e7156a8868fc&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:27.503403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.525088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09536-fd9f-406c-ab1e-8dcfdba6e224 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.530084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.564560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.565261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.565548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.565904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.566772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.567085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.568562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.585153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 510/1043] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:27.586193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:27.612165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b4375-3542-429d-ab93-a2527ab6a63c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.637579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:27.648738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.649447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.649738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.650274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.650972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.652353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4713f-b2c8-4112-b2f3-673961059f16 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.668910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 534/1044] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:15:27.703755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac1d5-4154-4136-8a2a-8d420bb83b61 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:27.748122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:27.748717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.749035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.749318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.750184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] RBAC: Adding query filter params (role.id=13eec5c3ca2440f6a6e219d4a1da1dd1, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:27.750502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:27.751974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3935760-a7f5-4a3b-948c-0881d015ea78 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:27.758614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 511/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:27 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:27.885177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c25a-d638-4b47-887f-43e3638227fd None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.885623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c25a-d638-4b47-887f-43e3638227fd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.886057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09c25a-d638-4b47-887f-43e3638227fd None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.888296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 535/1046] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:27.894893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:27.895232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:27.895473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:27.914281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:27.915499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:27.988079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 512/1047] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:28.012724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c25a-d638-4b47-887f-43e3638227fd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:28.110051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:28.125328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09c25a-d638-4b47-887f-43e3638227fd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:28.163195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:28.163994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:28.164279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:28.164613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:28.175220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:28.182967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:28.183878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:28.184718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 536/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:28.190151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:28.239776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:28.298997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fdf17-6a14-4beb-9b99-98305427755b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:28.335342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:28.336064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:28.336353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:28.336635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:28.337419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:28.337809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:28.339347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:28.352797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 513/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:28.354014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:28.459098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8adc8c6-deff-4760-a67e-ef3d5282e205 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:28.495884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:28.496822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:28.497175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:28.497463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] PATH_INFO: `/v3/users/1b9b2d04fa93416e8d69299e320f15d3` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:28.511920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] RBAC: Authorizing `identity:get_user(user_id=1b9b2d04fa93416e8d69299e320f15d3)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:28.513398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:28.521707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 537/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:28 2022] GET /identity/v3/users/1b9b2d04fa93416e8d69299e320f15d3 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:28.521852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:28.624374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60502b39-0176-4dc2-a9d5-d22edfa6d8de None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:28.626697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:28.659971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:28.660655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:28.660963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:28.661243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:28.734769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:28.736230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:28.843454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 514/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:15:28.849405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:28.858764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:28.957175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1e309-97b1-490a-a179-ed4f7816abab None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:28.992436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:28.993333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:28.993618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:28.993893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:28.995059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:28.996484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a4cb-ae44-4ebd-bade-34abfc8bce6b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:28.998407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 538/1052] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:28 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:29.004200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:29.008312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:29.106669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33dc92-3090-4c99-8a1a-23c76015c3df None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:29.140143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:29.140795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:29.141125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:29.141412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:29.142246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=680bfff994b248a5bdb4841d6c9b4058, user.id=1b9b2d04fa93416e8d69299e320f15d3) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:29.142566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:29.143993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d6127-8029-436b-b52d-8536721c363c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:29.149629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 515/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:29 2022] GET /identity/v3/role_assignments?user.id=1b9b2d04fa93416e8d69299e320f15d3&scope.project.id=680bfff994b248a5bdb4841d6c9b4058&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:29.829520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5c0-429b-4722-8aec-13a07e9694c5 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:29.830069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5c0-429b-4722-8aec-13a07e9694c5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:29.830069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5c0-429b-4722-8aec-13a07e9694c5 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:29.831553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 539/1054] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:29.837306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:29.837737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:29.837915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:29.855569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:29.856794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:29.922416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0dba-4073-4fe8-a1bf-20c4eb00c1f0 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:29.922713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0dba-4073-4fe8-a1bf-20c4eb00c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:29.923001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0dba-4073-4fe8-a1bf-20c4eb00c1f0 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:29.924519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 540/1055] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:29.925303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:29.929685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:29.929984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:29.930273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:29.937151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 516/1056] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:29.948309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:29.949529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:29.954672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.025317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 541/1057] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:30.046812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.050064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.067604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4c7a7-298c-45bc-81b7-871e79c51959 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.108038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.109061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.109352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.109937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] PATH_INFO: `/v3/services/image` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.110870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.112245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:30.126637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:30.127628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:30.128476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 517/1058] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Jun 16 00:15:29 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:30.134392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.151238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cf3e4-e6dc-48d6-a911-e872ff7006c2 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.187939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.188762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.189147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.189431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.201034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.201217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:30.209762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:30.210761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:30.211616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 542/1059] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:30.217029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.247498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c49a65-1854-495c-a3f2-e3c440cc630d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.285854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.286463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.286753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.287035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.287756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] RBAC: Adding query filter params (name=image) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:30.288069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.289473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.289945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.295039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 518/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:30.300945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.338302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87507f97-f9b2-4d7f-97d4-0fe6634c3da4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.374399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.375189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.375550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.375836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.376595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:30.376925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.378298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.387229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 543/1061] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:30.395236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.412408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed53e6-179d-47b1-ba18-8c8bef4bca38 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.439274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.450387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.451069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.451355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.451638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.452351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] RBAC: Adding query filter params (type=image) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:30.452664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.454074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.459799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 519/1062] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:30.465308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.506165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5c74-9591-4572-b1ab-d56eb216596c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.535516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.546513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.547981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.548481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.549012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.561003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.562672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.574747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36059-c99b-4701-9280-aeafa58ff33d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:30.581904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:30.582897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:30.583798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:30.583798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:30.583798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:30.583798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:30.583798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 544/1063] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:30.589332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.612075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.612735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.613204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.613550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.614296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.615673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.636770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 520/1064] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:15:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:30.723078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277ba41-3a82-4a84-aece-4b4cad5bec7f None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.770133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.770815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.771098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.771372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.772103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:30.772416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.773827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.787199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 545/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:30.792865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:30.847403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:30.907784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59266a3-8908-4e35-b3e2-0569b4d56057 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:30.942062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:30.943030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:30.943319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:30.943595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:30.951387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:30.952831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:30.967013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:30.967953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:30.973858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 521/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:30.975933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.012113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.078975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0647a-1aaa-41b2-8881-b689b02fed45 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.112189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.112784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.113694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.113971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.114814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:31.115122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.116542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da8714-4336-4366-8a21-83b2be65cce5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.138781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 546/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:15:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:31.140414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.242098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22ed62-aa00-4064-9179-3e6f1d9df8be None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.250529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.280793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.281621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.281908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.282186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.294729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.296175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.304723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc15ac-1912-4fb8-a472-363893399784 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.305076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc15ac-1912-4fb8-a472-363893399784 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.305367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc15ac-1912-4fb8-a472-363893399784 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.306748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 547/1068] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:31.311679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.311972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.312252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:31.315336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:31.316681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:31.317800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:31.317800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:31.317800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:31.317800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:31.320675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 522/1069] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:31.326585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.329991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:31.332013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:31.404553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 548/1070] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:31.427622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.439508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e1b48-7bb0-4be6-a3d8-8cd521e38219 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.452143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.481812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.482563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.482850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.483128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.484267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:31.484267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.485599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.497735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 523/1071] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:31.503655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.524129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.541222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cf754-d634-400b-8ca0-5b1d7e8a42ec None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.575744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.576541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.576830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.577148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.587330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:31.594885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:31.595925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:31.596959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:31.601907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 549/1072] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:31.602452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.619295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cb246-47f4-43f9-b4d9-269decfa6a5c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.656396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.657155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.657453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.657738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.665268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.666756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:31.681507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:31.682385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:31.683264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 524/1073] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:31.688298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.698144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.720570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864d67-b905-4bb2-b41a-d7a9644189bc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.760207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.760934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.761230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.761521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.762311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:31.762630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.764062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.773617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 550/1074] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:31.779730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.781143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.808945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e4d4-2137-484c-b7c0-1ac16cfb0009 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.846322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.847928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.848940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.849406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.850205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:31.850517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.851942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.865157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 525/1075] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:31.870855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.891623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0884c7d4-1dd9-4f94-b19e-3d6d527cb976 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:31.929990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:31.930788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:31.931074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:31.931357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] PATH_INFO: `/v3/users/1b9b2d04fa93416e8d69299e320f15d3` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:31.932543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9b2d04fa93416e8d69299e320f15d3)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:31.934029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:31.940136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 551/1076] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/users/1b9b2d04fa93416e8d69299e320f15d3 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:31.941523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:31.956387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:31.983152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0c3d9-63b5-4b36-a067-a8e632a6beda None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:32.018010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:32.018900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:32.019240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:32.019525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/13eec5c3ca2440f6a6e219d4a1da1dd1` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:32.021857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=d50db388db1446eeb60ccaa17ff55389, role_id=13eec5c3ca2440f6a6e219d4a1da1dd1)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:32.023279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:32.033096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:32.052909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 526/1077] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:15:31 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/d50db388db1446eeb60ccaa17ff55389/roles/13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:15:32.053121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0a43a-1bca-4675-9ebd-2a76a5e8a077 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:32.089222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:32.090165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:32.090451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:32.090729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:32.092017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:32.093690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd50fdb-6833-43ce-b433-25468963a026 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:32.101105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 552/1078] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:31 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:32.102820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:32.215491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe32e-8812-4848-be53-5cf2f72f3d34 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:32.250604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:32.251498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:32.251779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:32.252053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058/users/1b9b2d04fa93416e8d69299e320f15d3/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:32.254229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=680bfff994b248a5bdb4841d6c9b4058, user_id=1b9b2d04fa93416e8d69299e320f15d3, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:32.255650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f48632-5307-401e-aae5-c121712353b1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:32.270853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 527/1079] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:15:32 2022] PUT /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058/users/1b9b2d04fa93416e8d69299e320f15d3/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:15:32.835731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84a40-a066-4bb7-9926-f9d44ff3668c None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:32.836089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84a40-a066-4bb7-9926-f9d44ff3668c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:32.836312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef84a40-a066-4bb7-9926-f9d44ff3668c None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:32.837912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 553/1080] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:32.842987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:32.843292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:32.843579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:32.865427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:32.866650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:32.922245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:32.937907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 528/1081] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:32.961006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef84a40-a066-4bb7-9926-f9d44ff3668c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:33.044608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:33.066855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef84a40-a066-4bb7-9926-f9d44ff3668c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:33.100035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:33.100693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:33.101012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:33.101297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:33.186475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:33.187933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:33.267432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:33.291077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 554/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:33.296791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:33.398053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220451-b303-4a9f-9189-aa43bee36b95 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:33.431770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:33.432565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:33.432849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:33.433151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:33.444170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:33.445638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fb74d57f-ec9c-4cab-b284-30518133ab2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:33.462461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:33.463350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:33.469064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:33.469064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:33.469064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:33.469064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:33.469064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 529/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:33.469721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:33.572885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67e0ad-5335-4c62-af53-c35dd5de6209 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:33.606972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:33.607642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:33.607925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:33.608201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:33.608986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:33.609315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:33.610679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71d8f1-7450-4a40-a0d6-a3337f8bc054 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:33.628294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 555/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:34.224829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69999e5f-b85c-4bb5-b791-ec4e89ed2f1c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.225405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69999e5f-b85c-4bb5-b791-ec4e89ed2f1c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.225500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69999e5f-b85c-4bb5-b791-ec4e89ed2f1c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.226864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 530/1085] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:34.232358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723219-1172-4c07-88fa-a44acd31e0d0 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.232829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723219-1172-4c07-88fa-a44acd31e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.232982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb723219-1172-4c07-88fa-a44acd31e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.257229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-eb723219-1172-4c07-88fa-a44acd31e0d0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:34.258450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-eb723219-1172-4c07-88fa-a44acd31e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:34.294643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.328508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 556/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:34.352150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69999e5f-b85c-4bb5-b791-ec4e89ed2f1c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.403131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.465248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69999e5f-b85c-4bb5-b791-ec4e89ed2f1c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.466163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89da4-8fc7-4cdc-806b-c7d24bd19903 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.466455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89da4-8fc7-4cdc-806b-c7d24bd19903 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.466737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89da4-8fc7-4cdc-806b-c7d24bd19903 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.468111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 557/1087] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:34.474181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.474510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.474999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.492290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:34.493515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:34.500202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a681e-ccee-462b-a868-d32209089893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.501046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a681e-ccee-462b-a868-d32209089893 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.501353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a681e-ccee-462b-a868-d32209089893 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.501631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a681e-ccee-462b-a868-d32209089893 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.511868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a681e-ccee-462b-a868-d32209089893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-193a681e-ccee-462b-a868-d32209089893 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:34.520729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:34.521733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:34.522645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 531/1088] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:34.527984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a681e-ccee-462b-a868-d32209089893 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.566327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 558/1089] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:34.590881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.624347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.627537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a681e-ccee-462b-a868-d32209089893 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.667202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.667866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.668153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.668431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.669219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:34.669531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:34.670893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:34.678963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 532/1090] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:34.684305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.692685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.695923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0c8f7-5f84-4e0e-9fdf-2ec999fe4b72 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.735071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.735868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.736149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.736426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.747572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:34.755946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:34.757209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:34.758511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 559/1091] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:34.762982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.786860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83ca43-6a3d-4298-baa4-f5a9b7c05878 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.821940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.822742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.823030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.823313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.834900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:34.836350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:34.853453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:34.854459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:34.855349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:34.855349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:34.855349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:34.855349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:34.855349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 533/1092] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:34.860207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.865119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70391cd-ae7c-4ccb-90dd-0c490c145524 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.875812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.899559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.900490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.900781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.901100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.901847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:34.902160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:34.903526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:34.912460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 560/1093] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:34.918164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:34.942851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:34.960511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e66-be67-4330-838f-6c25adeb5480 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:34.994173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:34.994777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:34.995064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:34.995342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:34.996069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:34.996383 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:34.997782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.009730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 534/1094] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:35.015553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.017286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd2479-1a6e-4a45-8c28-481f119517e8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.051695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.052563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.052848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.053165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] PATH_INFO: `/v3/users/1b9b2d04fa93416e8d69299e320f15d3` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.054336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] RBAC: Authorizing `identity:get_user(user_id=1b9b2d04fa93416e8d69299e320f15d3)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.055753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.057415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 561/1095] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:34 2022] GET /identity/v3/users/1b9b2d04fa93416e8d69299e320f15d3 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:35.062942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.089666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.114873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dc8e7-3b71-4824-8da4-04e2e2eede22 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.147865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.148654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.148974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.149250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.152680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.156429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.157895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.162904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7e4c0-cc1b-4dc4-b76b-a6327170e329 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:15:35.172351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:35.173380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:35.174340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 535/1096] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:35.179577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.196651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.197355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.197640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.197919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.263594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.264994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.277599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365a597-d7d6-4510-8a67-d763cfc78be7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.311660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.312337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.312622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.312930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.313775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:35.314091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.315517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.317472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.334297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 536/1097] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:35.339799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.353180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.364906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 562/1098] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:15:35.369814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.436239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcacc-8e3b-472f-a318-42669b0f28d6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.467065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2425b52-999c-4dcd-9978-eb6b6e9d482c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.469272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.469915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.470199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.470473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.507833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.508683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.508993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.509271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.510830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.512596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.514847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 563/1099] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:35.521455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.530424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.541510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.542986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.582499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.626984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d46f9-6762-4877-88ed-1674fe516145 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.648444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 537/1100] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:15:35.654316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.664335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.665031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.665314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.665588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.666427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=680bfff994b248a5bdb4841d6c9b4058, user.id=1b9b2d04fa93416e8d69299e320f15d3) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:35.666730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.668169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.674561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 564/1101] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/role_assignments?user.id=1b9b2d04fa93416e8d69299e320f15d3&scope.project.id=680bfff994b248a5bdb4841d6c9b4058&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:15:35.756649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:35.767698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b4e7b-d5ed-4c06-a391-d9ce8f415f8f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.812928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:35.812928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:35.812928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:35.812928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:35.825276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:35.826858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:15:35.846556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:35.847796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:35.849016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:15:35.849016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:15:35.849016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:15:35.849016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:35.849016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 538/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:35.853687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:35.971820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f3f-43f2-48b7-97e2-4db01bbe77cc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:35.994857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:36.007291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:36.007965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:36.008257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:36.008541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:36.009310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:36.009625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:36.011002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:36.022383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 565/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:36.028440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:36.126278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed5d1e-3a5a-4b60-8ad0-46c0ee544189 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:36.160056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:36.160853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:36.161174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:36.161453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:36.168069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:36.169553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:15:36.183567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:36.184528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:15:36.185598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 539/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:15:36.190506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:36.287081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a192a-6301-48d6-a9ab-cbad5194833a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:36.320457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:36.321152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:36.321446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:36.321727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:36.322519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:36.322836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:36.324257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ebf7f-ac64-49f2-9393-959534732df5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:36.336090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 566/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:36.342195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:36.346553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:36.440347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8a4e7-c96c-4ea7-8aed-2d309064f91b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:36.473066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:36.473863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:36.474097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:36.474264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:36.475151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] RBAC: Adding query filter params (role.id=13eec5c3ca2440f6a6e219d4a1da1dd1, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=d50db388db1446eeb60ccaa17ff55389) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:36.475552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:36.477123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de51041-f2a8-40e4-8bb8-ac730a52614d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:36.483236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 540/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:36 2022] GET /identity/v3/role_assignments?user.id=d50db388db1446eeb60ccaa17ff55389&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=13eec5c3ca2440f6a6e219d4a1da1dd1 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:37.902361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bcaad-9cdd-4834-9a26-bfa7be7b1f05 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:37.902905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bcaad-9cdd-4834-9a26-bfa7be7b1f05 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:37.902905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bcaad-9cdd-4834-9a26-bfa7be7b1f05 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:37.904268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 567/1107] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:37.909323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:37.909798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:37.910090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:37.926759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:37.927942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:37.987292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:37.993006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 541/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:38.017442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bcaad-9cdd-4834-9a26-bfa7be7b1f05 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:38.060683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:38.115392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bcaad-9cdd-4834-9a26-bfa7be7b1f05 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:38.147681 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:38.148465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:38.148750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:38.149057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:38.150463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:38.151874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca89c0-6051-4090-9556-3b621a28ec13 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:38.153543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 568/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:38.159111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:38.255301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa98885-8992-4733-a1c1-3456f47fae38 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:38.289157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:38.289826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:38.290113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:38.290395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:38.291111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:38.292555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a05e3-acc9-40cc-8e38-44a8f37e9759 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:38.313708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 542/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:15:38 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:40.455793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691999bd-e0e0-43d8-aceb-63ecbb798aac None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.456838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691999bd-e0e0-43d8-aceb-63ecbb798aac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.456983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691999bd-e0e0-43d8-aceb-63ecbb798aac None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.458373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 569/1111] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:40.463545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.463924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.464105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.482043 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:40.483411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:40.532867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:40.551485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 543/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:40.575795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691999bd-e0e0-43d8-aceb-63ecbb798aac None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:40.675056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691999bd-e0e0-43d8-aceb-63ecbb798aac None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:40.685404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:40.708526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:40.709357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.709640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.709920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] PATH_INFO: `/v3/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.711373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] RBAC: Authorizing `identity:get_role(role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:40.712735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:40.714194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 570/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:40 2022] GET /identity/v3/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:40.720124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:40.818453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207c401-21a9-436a-b1e3-dcafab54f29e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:40.855717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:40.857259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:40.857748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:40.858256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] PATH_INFO: `/v3/users/6650af27ac814aae8adeea53ad460481` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:40.871344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] RBAC: Authorizing `identity:get_user(user_id=6650af27ac814aae8adeea53ad460481)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:40.872777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:40.874462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 544/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:40 2022] GET /identity/v3/users/6650af27ac814aae8adeea53ad460481 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:40.880765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:40.977258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecd749-ae04-4935-b2e7-f64661b433c0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:41.010625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:41.011262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:41.011547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:41.011825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:41.055422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:41.076453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:41.077917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:41.170608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 571/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:41.176562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:41.218167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:41.272061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2678c83-3f70-4778-b42b-fce46e58be4c None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:41.305037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:41.305886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:41.306169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:41.306446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:41.307602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:41.309046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147421c4-60f4-4536-ad46-3f6386645763 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:41.310699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 545/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:41 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:41.317760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:41.393226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:41.414955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32e778-316a-45ec-bcc6-f67bd98a4c5b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:41.448011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:41.448655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:41.448976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:41.449259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:41.450088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=680bfff994b248a5bdb4841d6c9b4058, user.id=6650af27ac814aae8adeea53ad460481) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:41.450402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:41.451830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fde634-d712-4bf4-947d-25daad5a2594 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:41.458060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 572/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:41 2022] GET /identity/v3/role_assignments?user.id=6650af27ac814aae8adeea53ad460481&scope.project.id=680bfff994b248a5bdb4841d6c9b4058&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:43.581947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f5fc4-8579-4231-bb1b-edcd27bb2a8a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.582528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f5fc4-8579-4231-bb1b-edcd27bb2a8a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.582528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f5fc4-8579-4231-bb1b-edcd27bb2a8a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.583855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 546/1118] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:15:43.589107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.589505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.589675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.606613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:43.607775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:43.673888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 573/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:43.695695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f5fc4-8579-4231-bb1b-edcd27bb2a8a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:43.795905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f5fc4-8579-4231-bb1b-edcd27bb2a8a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:43.806327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:43.829592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:43.830445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.830727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.831635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] PATH_INFO: `/v3/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.833296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] RBAC: Authorizing `identity:get_role(role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:43.834608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:43.835964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 547/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:43 2022] GET /identity/v3/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:43.841882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:43.909978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:43.938047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65192086-94c0-4e82-b154-3b1b25e6cf5b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:43.972431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:43.973328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:43.973610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:43.973881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] PATH_INFO: `/v3/users/6650af27ac814aae8adeea53ad460481` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:43.975041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=6650af27ac814aae8adeea53ad460481)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:43.976462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:43.978139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 574/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:43 2022] GET /identity/v3/users/6650af27ac814aae8adeea53ad460481 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:43.983810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:44.079601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d4053-fc0c-40ef-8bbb-372a050201be None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:44.113407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:44.114187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:44.114469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:44.114743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:44.115895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:44.117340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e5e78-44ce-4bda-aa92-ee8c47b6ff79 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:44.118991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 548/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:43 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:44.124710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:44.221222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f036f2-6dca-42c0-90c5-ae4df2dd8bd4 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:44.255038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:44.255909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:44.256192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:44.256468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058/users/6650af27ac814aae8adeea53ad460481/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:44.258619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] RBAC: Authorizing `identity:create_grant(project_id=680bfff994b248a5bdb4841d6c9b4058, user_id=6650af27ac814aae8adeea53ad460481, role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:44.260033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabdf9a-2bd1-4f7d-8276-82ad2f432e22 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:44.268449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:44.273066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 575/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:15:44 2022] PUT /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058/users/6650af27ac814aae8adeea53ad460481/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:15:46.366292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e544a1-cb89-445d-b9b1-c4afa7a67282 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.366830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e544a1-cb89-445d-b9b1-c4afa7a67282 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.366830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e544a1-cb89-445d-b9b1-c4afa7a67282 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.368205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 549/1124] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:46.373271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.373642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.373841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.390626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:46.391811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:46.456700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 576/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:46.479136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e544a1-cb89-445d-b9b1-c4afa7a67282 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.525752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.577468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e544a1-cb89-445d-b9b1-c4afa7a67282 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.609203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.610048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.610331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.610609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] PATH_INFO: `/v3/roles/dfddcd6381e14203ad13757e6d33fd79` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.612034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] RBAC: Authorizing `identity:get_role(role_id=dfddcd6381e14203ad13757e6d33fd79)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.614041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.615378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 550/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/roles/dfddcd6381e14203ad13757e6d33fd79 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:46.621136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.715960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be225f3-3a95-4f99-be74-70b2ef542f28 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.745863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.752022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.753520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.753813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.754093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] PATH_INFO: `/v3/users/6650af27ac814aae8adeea53ad460481` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.755275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] RBAC: Authorizing `identity:get_user(user_id=6650af27ac814aae8adeea53ad460481)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.756709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:46.758368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 577/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/users/6650af27ac814aae8adeea53ad460481 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:46.764017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:46.860901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d722fc9-8505-4fd1-80c6-1f1768e67748 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:46.875045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:46.895497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:46.896148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:46.896432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:46.896708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:46.961067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:46.962504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:47.057153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 551/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:47.063199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:47.161130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214855ce-8c69-4d66-ad29-da87eb42b279 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:47.193799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:47.194649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:47.194938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:47.195219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] PATH_INFO: `/v3/projects/680bfff994b248a5bdb4841d6c9b4058` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:47.196375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] RBAC: Authorizing `identity:get_project(project_id=680bfff994b248a5bdb4841d6c9b4058)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:47.197827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61388e0-64c1-49bf-8fc5-38d6b5fbe3cf None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:47.199501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 578/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:47 2022] GET /identity/v3/projects/680bfff994b248a5bdb4841d6c9b4058 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:47.205562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:47.220626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:47.302065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95965f5e-41b8-495b-a599-15088826adf3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:47.334475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:47.335114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:47.335397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:47.335674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:47.336496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] RBAC: Adding query filter params (role.id=dfddcd6381e14203ad13757e6d33fd79, scope.project.id=680bfff994b248a5bdb4841d6c9b4058, user.id=6650af27ac814aae8adeea53ad460481) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:47.336808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:47.338252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc103423-f88e-4879-8d84-d02a1863fe21 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:47.343852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 552/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:47 2022] GET /identity/v3/role_assignments?user.id=6650af27ac814aae8adeea53ad460481&scope.project.id=680bfff994b248a5bdb4841d6c9b4058&role.id=dfddcd6381e14203ad13757e6d33fd79 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:49.470565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc547f6-3acf-4ff2-a675-815049a28dac None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.471097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc547f6-3acf-4ff2-a675-815049a28dac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.471190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc547f6-3acf-4ff2-a675-815049a28dac None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.472520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 579/1131] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:49.478036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.478479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.478612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.495424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:49.496608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:49.561106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 553/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:49.584351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc547f6-3acf-4ff2-a675-815049a28dac None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.657784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:49.692439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc547f6-3acf-4ff2-a675-815049a28dac None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.725496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.726749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.726749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.726749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.727899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:49.729370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb003486-2b3c-495a-8b03-b49faddca5dd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:49.730978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 580/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:49.736834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:49.747206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:49.835879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70235f-45ab-4621-9395-c7caa2b425a3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:49.868582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:49.869267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:49.869552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:49.869825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:49.870532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:49.871946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3832e6-c65d-464c-aacc-1344a7f2b04b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:49.896000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 554/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:15:49 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:15:52.071076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4850-1103-4b50-ab60-cd59df8e1208 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:52.071579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4850-1103-4b50-ab60-cd59df8e1208 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:52.071671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4850-1103-4b50-ab60-cd59df8e1208 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:52.073040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 581/1135] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:52.078262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:52.078640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:52.078833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:52.095381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:52.097150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:52.150008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:52.162664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 555/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:15:52.186924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4850-1103-4b50-ab60-cd59df8e1208 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:52.285504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4850-1103-4b50-ab60-cd59df8e1208 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:52.334600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:52.335331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:52.335619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:52.335899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:52.337088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:52.338634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726998ea-429d-4c6c-8bfc-7c0a4b062159 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:52.340302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 582/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:15:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:52.350213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:52.445299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd07391-ff0f-4233-8003-4b8c3a91e0ff None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:52.478064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:52.478738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:52.479024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:52.479300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:52.480008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:52.481502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08cda2f-786f-4a18-8c43-16cb15d41d95 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:52.501348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 556/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:15:52 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:15:54.620336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22ac4-b7bb-43cc-bf48-4d84f4043cd5 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:54.620867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22ac4-b7bb-43cc-bf48-4d84f4043cd5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:54.620984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22ac4-b7bb-43cc-bf48-4d84f4043cd5 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:54.622296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 583/1139] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:54.627576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:54.627984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:54.628137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:54.644271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:54.645468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:54.686457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:54.711333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 557/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:54.735072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba22ac4-b7bb-43cc-bf48-4d84f4043cd5 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:54.738026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:54.834100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba22ac4-b7bb-43cc-bf48-4d84f4043cd5 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:54.867093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:54.867871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:54.868154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:54.868427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:54.878179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:54.885584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:54.886637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:54.887666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 584/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:54.892851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:54.987512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7ad73-836f-428f-94b6-194bc02a86cc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:55.020984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:55.021654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:55.021941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:55.022226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:55.022981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:55.023295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:55.024932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:55.032970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 558/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:55.039933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:55.100696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:55.138862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a66b41-d4a1-46ba-828c-ef3edb50a483 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:55.171169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:55.172025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:55.172308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:55.172587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] PATH_INFO: `/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:55.185214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] RBAC: Authorizing `identity:get_user(user_id=bd63d1fd71a04d56ac13d32c8435b4ad)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:55.186640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:55.188263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 585/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:55 2022] GET /identity/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:55.194535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:55.233334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:55.290723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bfab9-7bf1-47c2-9e3f-334c55981088 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:55.322663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:55.323307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:55.323591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:55.323872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:55.389187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:55.390556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:55.427147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:55.486342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 559/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:15:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:15:55.493339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:55.588985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317efc5-b109-439c-8295-114026d37ae2 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:55.608079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:55.622163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:55.623036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:55.623358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:55.623641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] PATH_INFO: `/v3/projects/00793209221c45878797b060214d8102` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:55.624811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] RBAC: Authorizing `identity:get_project(project_id=00793209221c45878797b060214d8102)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:55.626290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdaf7a5-701a-4ef7-84b0-c55422132eea None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:55.628510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 586/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:55 2022] GET /identity/v3/projects/00793209221c45878797b060214d8102 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:55.634590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:55.729431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b280e5-0ed6-47d5-813f-184521ae711e None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:55.761346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:55.761990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:55.762276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:55.762550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:55.763372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=00793209221c45878797b060214d8102, user.id=bd63d1fd71a04d56ac13d32c8435b4ad) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:55.763687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:55.765126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2427149-81aa-4f86-81ca-d2af758ac257 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:55.770944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 560/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:15:55 2022] GET /identity/v3/role_assignments?user.id=bd63d1fd71a04d56ac13d32c8435b4ad&scope.project.id=00793209221c45878797b060214d8102&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:57.849592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19110af-9a31-47d0-84df-50a42541254a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:57.850112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19110af-9a31-47d0-84df-50a42541254a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:57.850200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19110af-9a31-47d0-84df-50a42541254a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:57.851517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 587/1147] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:15:57.856673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:57.857129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:57.857272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:57.862414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:57.874918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:15:57.876112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:15:57.941280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 561/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:15:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:15:57.964129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19110af-9a31-47d0-84df-50a42541254a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:58.062920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19110af-9a31-47d0-84df-50a42541254a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:58.097333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:58.098124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:58.098413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:58.098693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:58.108453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:15:58.116248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:15:58.117379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:15:58.118274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 588/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:15:58.123562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:58.218371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352748-a8e5-474a-846d-42a003570fe3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:58.246392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:58.251464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:58.252651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:58.252970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:58.253251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:58.253985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:15:58.254286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:58.255635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:58.263424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 562/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:58.269653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:58.316984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:58.364934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b306ab-ae16-4fa1-a5f4-22c997a329ca None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:58.397162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:58.398012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:58.398294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:58.398567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] PATH_INFO: `/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:58.399970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] RBAC: Authorizing `identity:get_user(user_id=bd63d1fd71a04d56ac13d32c8435b4ad)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:58.401414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:58.403013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 589/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:15:58 2022] GET /identity/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:58.408779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:58.504639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc82e0-2801-4b5c-82a7-807babbd95ac None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:58.536362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:58.537234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:58.537520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:58.537797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] PATH_INFO: `/v3/projects/00793209221c45878797b060214d8102` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:58.538952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] RBAC: Authorizing `identity:get_project(project_id=00793209221c45878797b060214d8102)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:58.540361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e59bb3-ed56-42ac-a3b7-100d3a62991b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:58.542074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 563/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:15:58 2022] GET /identity/v3/projects/00793209221c45878797b060214d8102 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:15:58.547824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:15:58.643196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba04f28-c9cc-4537-be4d-18365fe85660 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:15:58.663481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:15:58.680473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:15:58.681386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:15:58.681669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:15:58.681941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] PATH_INFO: `/v3/projects/00793209221c45878797b060214d8102/users/bd63d1fd71a04d56ac13d32c8435b4ad/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:15:58.683844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] RBAC: Authorizing `identity:create_grant(project_id=00793209221c45878797b060214d8102, user_id=bd63d1fd71a04d56ac13d32c8435b4ad, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:15:58.685290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99faf5d-0e89-4b6a-ba64-6fae20ffa690 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:15:58.698752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 590/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:15:58 2022] PUT /identity/v3/projects/00793209221c45878797b060214d8102/users/bd63d1fd71a04d56ac13d32c8435b4ad/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:16:00.818080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185759-e47c-4ccd-8941-e587dfcf1ab7 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.818604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185759-e47c-4ccd-8941-e587dfcf1ab7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.818721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7185759-e47c-4ccd-8941-e587dfcf1ab7 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.819998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 564/1154] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:00.825112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:00.825522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:00.825688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:00.844095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:00.845347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:00.911176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 591/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:00.934525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7185759-e47c-4ccd-8941-e587dfcf1ab7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:00.995834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.035284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7185759-e47c-4ccd-8941-e587dfcf1ab7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.067858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.068640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.068952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.069236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.079441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:01.087166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:01.088153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:16:01.089018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 565/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:01.094287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.190549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ec48b-17ed-42f5-8f76-077bae8262fd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.197508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.224645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.225333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.225622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.225899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.226649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:01.226958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.228317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.236121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 592/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.242172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.341014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678dede-310f-40eb-9e53-55f8987297b1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.363290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.374518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.375379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.375665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.375944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] PATH_INFO: `/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.378226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] RBAC: Authorizing `identity:get_user(user_id=bd63d1fd71a04d56ac13d32c8435b4ad)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.379660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.381302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 566/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/users/bd63d1fd71a04d56ac13d32c8435b4ad => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.386936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.482874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aac11-9cc0-477e-ab62-e1eeea468edc None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.515095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.515730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.516013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.516288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.549455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.580779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.582254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.676148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 593/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:16:01.681957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.780039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975bd77-269a-4005-bc90-bd1955b4d8b1 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.813104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.813958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.814249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.815165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] PATH_INFO: `/v3/projects/00793209221c45878797b060214d8102` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.816409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=00793209221c45878797b060214d8102)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.817877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40015af3-877a-48f8-8a8a-4edca55fd4f9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.819576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 567/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/projects/00793209221c45878797b060214d8102 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:01.825636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:01.888601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:01.921216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ec46d-a10b-4e3d-a01e-f5f478466a22 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:01.953460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:01.954046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:01.954329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:01.954604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:01.955570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=00793209221c45878797b060214d8102, user.id=bd63d1fd71a04d56ac13d32c8435b4ad) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:01.955886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:01.957332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78a5ef-894b-421f-b68b-4dec0194a3f3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:01.962939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 594/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:01 2022] GET /identity/v3/role_assignments?user.id=bd63d1fd71a04d56ac13d32c8435b4ad&scope.project.id=00793209221c45878797b060214d8102&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:04.090864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89058654-086e-4edd-8b08-dddf32dbd79a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.091391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89058654-086e-4edd-8b08-dddf32dbd79a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.091478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89058654-086e-4edd-8b08-dddf32dbd79a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.092800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 568/1162] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:04.099176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.099626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.099717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.116468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:04.117953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:04.183219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 595/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:04.205945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89058654-086e-4edd-8b08-dddf32dbd79a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:04.263846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:04.306038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89058654-086e-4edd-8b08-dddf32dbd79a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:04.338456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:04.339099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.339389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.339669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.404329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:04.405739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:04.472534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:04.513687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 569/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:16:04.520634 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:04.578741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:04.616751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0729e8c6-bdc3-4ba9-bbdf-153d3697b4c0 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:04.649106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:04.650153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.650438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.650711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.657204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:04.659297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-12953f69-639a-41ac-b499-7bbe69d488ae None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:04.673010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:04.674254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:16:04.675182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 596/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:16:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:04.680204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:04.776682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202c8-1829-4795-9380-e5eb302226f0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:04.825820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:04.826499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:04.826785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:04.827257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:04.827986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:04.828285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:04.829878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f51899-027e-45f5-b2be-5cc90a9d34a4 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:04.838397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 570/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Jun 16 00:16:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:06.942911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2490777-82f2-4728-a3a2-9923f790881a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.943510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2490777-82f2-4728-a3a2-9923f790881a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.943510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2490777-82f2-4728-a3a2-9923f790881a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.944848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 597/1167] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:06.951003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405efb8-8e2a-4cc7-a7ab-bfd6ec64b6af None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:06.951437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405efb8-8e2a-4cc7-a7ab-bfd6ec64b6af None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:06.951563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405efb8-8e2a-4cc7-a7ab-bfd6ec64b6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:06.969236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a405efb8-8e2a-4cc7-a7ab-bfd6ec64b6af None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:06.970437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a405efb8-8e2a-4cc7-a7ab-bfd6ec64b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:06.987233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.036550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 571/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:07.059755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2490777-82f2-4728-a3a2-9923f790881a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:07.158496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2490777-82f2-4728-a3a2-9923f790881a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:07.179259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:07.190995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:07.191642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:07.191928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:07.192207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:07.192968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:07.194325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716a307-6643-4794-8937-f0e103220dd3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:07.208647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 598/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Thu Jun 16 00:16:07 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:09.334879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c7e79-b9d8-4ba0-b108-affe3e9e9a8b None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.335426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c7e79-b9d8-4ba0-b108-affe3e9e9a8b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.335426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c7e79-b9d8-4ba0-b108-affe3e9e9a8b None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.336795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 572/1170] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:09.342095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.342480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.342672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.359862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:09.361678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:09.426263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 599/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:09.449596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c7e79-b9d8-4ba0-b108-affe3e9e9a8b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:09.553157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c7e79-b9d8-4ba0-b108-affe3e9e9a8b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:09.586295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:09.587094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.587377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.587651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] PATH_INFO: `/v3/domains/688e4334b52c4f7ebea3d9db3d4a2db2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.598864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=688e4334b52c4f7ebea3d9db3d4a2db2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:09.600298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eadbf-63a4-4e9b-b158-2f8b4a61d886 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:09.601952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 573/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Jun 16 00:16:09 2022] GET /identity/v3/domains/688e4334b52c4f7ebea3d9db3d4a2db2 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:09.607792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:09.702933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018f213-712f-4b5f-b61b-c6ac1496aba4 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:09.716344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:09.738140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:09.738811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:09.739102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:09.739384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:09.740089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:09.741548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb0c81-9f3a-44f7-969e-e78f48b512d2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:09.765911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 600/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:16:09 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:11.915786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e652-b8c1-40e3-b73c-de9c2f9cc881 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.916344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e652-b8c1-40e3-b73c-de9c2f9cc881 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.916344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e652-b8c1-40e3-b73c-de9c2f9cc881 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.917763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 574/1174] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:11.922976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:11.923391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:11.923600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:11.941677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:11.942859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:12.007947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 601/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:12.033705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e652-b8c1-40e3-b73c-de9c2f9cc881 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.048681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:12.133062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e652-b8c1-40e3-b73c-de9c2f9cc881 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.166041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.166899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.167188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.167469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] PATH_INFO: `/v3/domains/688e4334b52c4f7ebea3d9db3d4a2db2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.168641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=688e4334b52c4f7ebea3d9db3d4a2db2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:12.170179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f81d04-012c-42b1-b2df-75daa0abdca7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:12.171784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 575/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Jun 16 00:16:12 2022] GET /identity/v3/domains/688e4334b52c4f7ebea3d9db3d4a2db2 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:12.177421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:12.265295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:12.272816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96e018-95b7-4c9d-aab1-e872a92c7e82 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:12.306761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:12.307427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:12.307710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:12.307984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:12.308692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:12.310159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5a91d-6717-4fe3-bc0f-6c1eedffe6c1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:12.329358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 602/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:16:12 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:16:14.452107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f7961-9dc2-41b2-87cb-6bfedf073ecf None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.452679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f7961-9dc2-41b2-87cb-6bfedf073ecf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.452789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f7961-9dc2-41b2-87cb-6bfedf073ecf None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.454278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 576/1178] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:14.460045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.460468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.460610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.478753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:14.479978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:14.546268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 603/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:14.570572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f7961-9dc2-41b2-87cb-6bfedf073ecf None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.649948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.669608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f7961-9dc2-41b2-87cb-6bfedf073ecf None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.701635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.702462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.702699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.702978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.713625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:14.721892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:14.723068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:16:14.723959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 577/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:14.729039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.809692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:14.828645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baf258-23b6-4ab4-997a-25982f634ff1 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:14.862089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:14.862757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:14.863051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:14.863335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:14.864094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:14.864412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:14.865813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:14.873680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 604/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:14.879804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:14.978627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f2b16-b8e2-4a36-9fbb-0586f6d65435 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.011450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.012318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.012605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.012907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] PATH_INFO: `/v3/users/3acb94358e834a33ac61982ec88fae20` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.024980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.028650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] RBAC: Authorizing `identity:get_user(user_id=3acb94358e834a33ac61982ec88fae20)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.030366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.032215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 578/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:14 2022] GET /identity/v3/users/3acb94358e834a33ac61982ec88fae20 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.038742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.135328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d85f7-714d-4801-a966-13aa13467984 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.169010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.169659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.169947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.170227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.181907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.237566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.238946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.335120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 605/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:16:15.340969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.439747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6612f0-92c5-4383-9eab-1826ad1415a6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.472900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.473768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.474053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.474330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] PATH_INFO: `/v3/projects/b5322a0122c1492ba0a61e953982550f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.475505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] RBAC: Authorizing `identity:get_project(project_id=b5322a0122c1492ba0a61e953982550f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.476953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267c8d-8fea-4fbe-96fb-22100504fd0e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.478617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 579/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/projects/b5322a0122c1492ba0a61e953982550f => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:15.484504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:15.526877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:15.581717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6f3d9-c83d-46c2-bfc0-931e6213b8c1 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:15.615145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:15.615790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:15.616073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:15.616352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:15.617202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=b5322a0122c1492ba0a61e953982550f, user.id=3acb94358e834a33ac61982ec88fae20) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:15.617521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:15.618937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62793fe8-525e-4cf6-bcd1-857370b04991 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:15.624190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 606/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:15 2022] GET /identity/v3/role_assignments?user.id=3acb94358e834a33ac61982ec88fae20&scope.project.id=b5322a0122c1492ba0a61e953982550f&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:17.705328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4416-7d49-4c17-b321-e4d293498768 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.705860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4416-7d49-4c17-b321-e4d293498768 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.705860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4416-7d49-4c17-b321-e4d293498768 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.707879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 580/1186] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:17.713037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.713425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.713605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.731075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:17.732258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:17.797478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 607/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:17.821177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa4416-7d49-4c17-b321-e4d293498768 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:17.886148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:17.922478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa4416-7d49-4c17-b321-e4d293498768 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:17.956498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:17.957254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:17.957542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:17.958084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:17.967880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:17.975536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:17.976592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:16:17.977592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 581/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:17.982770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.016826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:18.079856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d7309-2372-4cee-9993-7ead07d02b51 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:18.115366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:18.116043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.116332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.116611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.117413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:18.117727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:18.119097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:18.127813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 608/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:18.133748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.229820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad088beb-5e73-4ae4-92ed-9be7ba0e3ca9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:18.254827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:18.263530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:18.264393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.264684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.264994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] PATH_INFO: `/v3/users/3acb94358e834a33ac61982ec88fae20` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.266185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] RBAC: Authorizing `identity:get_user(user_id=3acb94358e834a33ac61982ec88fae20)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:18.267814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:18.269456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 582/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:18 2022] GET /identity/v3/users/3acb94358e834a33ac61982ec88fae20 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:18.275396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.368467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:18.372491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940813e-1c60-4dee-bd14-0c43bbf7d780 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:18.405615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:18.406473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.406759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.407037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] PATH_INFO: `/v3/projects/b5322a0122c1492ba0a61e953982550f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.408790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] RBAC: Authorizing `identity:get_project(project_id=b5322a0122c1492ba0a61e953982550f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:18.410238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69166-7cca-4e8e-9dde-48562fe208cd None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:18.411911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 609/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:18 2022] GET /identity/v3/projects/b5322a0122c1492ba0a61e953982550f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:18.417819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:18.516380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f543b-74e0-49eb-b397-405cc681c05f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:18.549051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:18.549916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:18.550198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:18.550474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] PATH_INFO: `/v3/projects/b5322a0122c1492ba0a61e953982550f/users/3acb94358e834a33ac61982ec88fae20/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:18.552396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5322a0122c1492ba0a61e953982550f, user_id=3acb94358e834a33ac61982ec88fae20, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:18.553847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242eda0-4edf-422c-882a-c19361deaab7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:18.566145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 583/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:16:18 2022] PUT /identity/v3/projects/b5322a0122c1492ba0a61e953982550f/users/3acb94358e834a33ac61982ec88fae20/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:16:20.686502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95847e9-1af2-4554-8e1f-b2ca7c06a14e None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.687063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95847e9-1af2-4554-8e1f-b2ca7c06a14e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.687063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95847e9-1af2-4554-8e1f-b2ca7c06a14e None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.688991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 610/1193] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:16:20.694210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.694583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.694784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.712428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:16:20.713651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:16:20.761733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:20.780484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 584/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:16:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:16:20.804772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95847e9-1af2-4554-8e1f-b2ca7c06a14e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:20.905845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95847e9-1af2-4554-8e1f-b2ca7c06a14e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:20.937664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:20.938887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:20.939191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:20.939507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:20.949734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:16:20.957909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:16:20.958979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:16:20.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 611/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:16:20.965054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.063943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a17a8-09dc-4018-8494-d448dc854ffa None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.098309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.098967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.099248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.099519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.100255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:21.100560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.101957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.109706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 585/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:21.115560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.142211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:21.212344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5632e0-4e3c-4074-a558-9713560b3c0e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.244823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.245724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.246011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.246287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] PATH_INFO: `/v3/users/3acb94358e834a33ac61982ec88fae20` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.247439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] RBAC: Authorizing `identity:get_user(user_id=3acb94358e834a33ac61982ec88fae20)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.248852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.250471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 612/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Jun 16 00:16:21 2022] GET /identity/v3/users/3acb94358e834a33ac61982ec88fae20 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:21.256201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.302266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:21.353762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b313700-f0f4-4cad-8b25-329c76975d6f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.385723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.386363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.386648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.386924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.452833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.454498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.498042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:21.549767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 586/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:16:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:16:21.555588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.644753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:21.651879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b6293-3297-48d1-964a-1e807d45455a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.686248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.687093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.687619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.687742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] PATH_INFO: `/v3/projects/b5322a0122c1492ba0a61e953982550f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.688955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] RBAC: Authorizing `identity:get_project(project_id=b5322a0122c1492ba0a61e953982550f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.690390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4d43-e3e0-413d-990b-34f9218b380d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.692074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 613/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Jun 16 00:16:21 2022] GET /identity/v3/projects/b5322a0122c1492ba0a61e953982550f => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:16:21.698048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:16:21.793998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d164b-748a-4c35-8a9c-fe90bace82c9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:16:21.832423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:16:21.833179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:16:21.833465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:16:21.833743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:16:21.834570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=b5322a0122c1492ba0a61e953982550f, user.id=3acb94358e834a33ac61982ec88fae20) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:16:21.834884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:16:21.836302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb16e2-05de-40d1-bcca-d4f179e437cf None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:16:21.842123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 587/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:16:21 2022] GET /identity/v3/role_assignments?user.id=3acb94358e834a33ac61982ec88fae20&scope.project.id=b5322a0122c1492ba0a61e953982550f&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:16:31.682655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=19 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:16:31.839854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:18.716907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab35e0-0804-4a7e-b8f2-e77fd348d45d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.716907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab35e0-0804-4a7e-b8f2-e77fd348d45d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.716907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab35e0-0804-4a7e-b8f2-e77fd348d45d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.718174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 614/1201] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:18.722600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:18.723046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:18.723204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:18.742704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:18.744046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:18.834315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 588/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:18.862141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab35e0-0804-4a7e-b8f2-e77fd348d45d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:18.998564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab35e0-0804-4a7e-b8f2-e77fd348d45d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:19.044301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:19.046019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:19.046500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:19.046773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:19.058613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:19.060497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:19.084750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:19.086110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:19.087311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:19.087311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:19.087311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:19.087311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:19.087311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 615/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:19.090969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:19.218458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051df3bf-5e30-4063-a2f6-a956b14a727a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:19.265419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:19.272771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:19.273881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:19.274234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:19.274614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:19.275512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:19.275967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:19.277549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3e34-48c4-4d9d-b847-ffac4815949c None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:19.298841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 589/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:19.304585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:19.382691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:19.478846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894919-a3e3-405f-8197-981862ea2f38 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:19.543914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:19.546088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:19.546909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:19.547689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:19.549309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:19.552345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8b1a0-f0eb-478c-98a2-6bcbb97edda6 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:19.583605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 616/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Jun 16 00:17:19 2022] POST /identity/v3/users => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:22.110810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c6fe6-c157-413d-a00b-3c5f36eedee5 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.111383 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c6fe6-c157-413d-a00b-3c5f36eedee5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.111383 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c6fe6-c157-413d-a00b-3c5f36eedee5 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.113304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 590/1206] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:22.119128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545cd9-c293-4145-985b-7737967b612d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.119579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545cd9-c293-4145-985b-7737967b612d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.119744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50545cd9-c293-4145-985b-7737967b612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.138697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-50545cd9-c293-4145-985b-7737967b612d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:22.140040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-50545cd9-c293-4145-985b-7737967b612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:22.227628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 617/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:22.249992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c6fe6-c157-413d-a00b-3c5f36eedee5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.363689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c6fe6-c157-413d-a00b-3c5f36eedee5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.406256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.407452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.408055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.408319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.420340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:22.428209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:22.429403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:22.430263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 591/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:22.437053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545cd9-c293-4145-985b-7737967b612d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.489766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.563805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50545cd9-c293-4145-985b-7737967b612d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.611514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.612505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.612808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.613195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.614089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:22.614390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.616064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.632553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 618/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:22.633860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.695384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.760844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d4fd-b041-489d-917b-fa2d3e7fd19a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:22.800772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:22.801605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:22.801909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:22.802189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:22.814671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:22.816223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:22.837848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:22.839184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:22.840769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:22.840769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:22.840769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:22.840769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:22.843419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 592/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:22.844766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:22.971857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:22.974605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe0085-f408-472b-acf2-be60f50f361c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.016164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.017430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.018937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.018937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.019130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.019364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.021237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.037855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 619/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.043835 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.166685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93756d69-27ee-4786-b85c-e3504d680246 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.168450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.208902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.209829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.210123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.210407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.218862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.220431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:23.237153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:23.238250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:23.239229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:23.243283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 593/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:23.244701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.365785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa71c01-0a0f-4ba1-b925-b7dd412e201c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.412932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.413903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.414252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.414729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.415950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:23.416452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.418563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.441450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 620/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:23.447363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.577040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6a894-07ab-4953-a6f2-e38456ad1af6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:23.622190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:23.623697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:23.624192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:23.624492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:23.662998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.711388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:23.715736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:23.842608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 594/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:23.844133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:23.864677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:23.957552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11049b93-8845-49d3-8875-3bde4b78d62e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.006659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.007461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.007742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.008021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.020040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.021529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:24.040974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:24.041972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:24.042865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:24.042865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:24.042865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:24.042865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:24.048924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 621/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:24.049091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.124454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.169506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fa7f7c-37ae-421a-93f3-56c7f70f932d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.205900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.206578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.206868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.207144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.207902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.208209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.209730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.223175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 595/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:24.229101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.313429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.354720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c0955-d444-490a-b135-9fa5b679ecdf None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.397603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.398582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.398867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.399145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.406572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.408017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:24.423438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:24.424730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:24.425673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 622/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:24.430634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.554147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff198e0-d575-49f9-a6fc-de3d92b4bed7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.579517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.594554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.595370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.595766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.596128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.597065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.597436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.599133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c1ca3-c6ff-4315-bbcc-8eb28ab1e94e None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.614978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 596/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:24.622154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:24.753497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fcbc3-485f-4181-9455-d9eee30d6eba None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:24.797526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:24.804189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:24.804864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:24.805185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:24.805463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:24.806445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=2add2dc7d84c497a937723b3badcc881) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:24.806885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:24.808504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d057a00-7162-4a55-b5d1-3bf5176fcd6f None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:24.815066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 623/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:24 2022] GET /identity/v3/role_assignments?user.id=2add2dc7d84c497a937723b3badcc881&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:27.091659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83409996-ba3f-417f-911a-de05ef5df4fc None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.092245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83409996-ba3f-417f-911a-de05ef5df4fc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.092245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83409996-ba3f-417f-911a-de05ef5df4fc None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.094019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 597/1220] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:27.100645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.101211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.101211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.123273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:27.125358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:27.208176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 624/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:27.232329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83409996-ba3f-417f-911a-de05ef5df4fc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.345637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83409996-ba3f-417f-911a-de05ef5df4fc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.384633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22300-492c-479d-a730-09df924f3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.385571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22300-492c-479d-a730-09df924f3029 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.385876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22300-492c-479d-a730-09df924f3029 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.386249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22300-492c-479d-a730-09df924f3029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.402243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df22300-492c-479d-a730-09df924f3029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-7df22300-492c-479d-a730-09df924f3029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:27.414063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.416171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:27.418288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 598/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:27.422889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.551449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10162d18-edba-41e8-aaaf-b9dd516fa023 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.583384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.596873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.597582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.597915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.598236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.599069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] RBAC: Adding query filter params (name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:27.599427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.601165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.613163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 625/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:27.620459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22300-492c-479d-a730-09df924f3029 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.676548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:27.749005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22300-492c-479d-a730-09df924f3029 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.786066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.786889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.787169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.787460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.800275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.801763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:27.826349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:27.827472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:27.828781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:27.828781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:27.828781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:27.828781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:27.828781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 599/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:27.833906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:27.944086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66e1d2-48b6-4f23-99f6-745939880008 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:27.984625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:27.985477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:27.985903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:27.986194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:27.986960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:27.987408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:27.988983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.001721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 626/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:28.007785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.125229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.133059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409df91-6d79-4519-ac84-c71daa6744b5 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.170104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.170937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.171222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.171496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.180463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.182072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:28.199049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.201242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:28.202333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 600/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.207644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.260905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.319493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35790997-eb96-45d6-908d-3534a79053c5 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.362486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.363261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.363555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.363849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.364784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:28.365149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.366672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.385030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 627/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:28.390936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.502529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1772da-9f6e-476c-bdf9-8700ddefbc8a None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.548000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.549015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.549433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.549719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.564310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.566075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:28.586137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.587632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.588804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:28.588804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:28.588804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:28.588804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:28.588804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 601/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.595895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.717506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75777d98-6b5e-40bf-8062-380662899efb None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.719906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.758864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.759601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.759997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.760282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.761210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:28.761658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.763954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.778082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 628/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:28.784955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:28.797256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:28.902770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ab9a9-2c26-4657-916d-08022241095d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:28.943025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:28.943889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:28.944174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:28.944450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:28.951856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:28.953560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:28.969307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:28.970411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:28.971542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 602/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:28.976800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.088790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af5014-75c7-4c89-b89e-ebfca4cca808 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.129046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.129721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.130008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.130288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.131087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:29.131401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.132826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5509b-bf4a-416a-a36b-dfc1cf10162c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.146422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 629/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:29.153787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:29.226763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:29.263844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c55f3-f34b-4b30-bbd6-2a155959d7eb None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:29.302732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:29.303828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:29.304154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:29.304555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/2add2dc7d84c497a937723b3badcc881/roles/34a2aafa93ee4002bfed71c1f3acca43` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:29.325268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=2add2dc7d84c497a937723b3badcc881, role_id=34a2aafa93ee4002bfed71c1f3acca43)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:29.327124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceffe87-0dc4-4432-8711-7fa2d299d20d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:29.341368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 603/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:29 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/2add2dc7d84c497a937723b3badcc881/roles/34a2aafa93ee4002bfed71c1f3acca43 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:31.667726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001542a-c64f-44eb-9072-ceaca1a70118 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.668298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001542a-c64f-44eb-9072-ceaca1a70118 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.668298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001542a-c64f-44eb-9072-ceaca1a70118 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.670322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 630/1233] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:31.676159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.676644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.676815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.695746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:31.696996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:31.781969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 604/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:31.806029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001542a-c64f-44eb-9072-ceaca1a70118 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:31.835445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:31.926281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001542a-c64f-44eb-9072-ceaca1a70118 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:31.963160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:31.963973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:31.964265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:31.964550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] PATH_INFO: `/v3/roles/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:31.975738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:31.985222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:31.986314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:31.987322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 631/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:31.993727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.046136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.118468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1da93-9573-48e0-8f97-b92efe5c3c18 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.153952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.154638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.154927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.155206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.155956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.156270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.157671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.165920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 605/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Jun 16 00:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:32.172313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.260518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.280100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f56c0-6398-4ac5-88ac-53cb95a453af None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.320107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.320962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.321348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.321581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.334749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.336375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:32.360323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:32.362279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:32.364214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:32.364214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:32.364214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:32.364214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:32.364214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 632/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:32.368582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.491422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1e322-8b5c-4ee3-8d00-35906004cca3 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.503435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.537757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.538548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.538880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.539264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.540098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.540432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.542042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.561130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 606/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:32.567006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.689368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db24a00-c8b7-4a6a-92d9-ad69bd647976 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.729460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.730306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.730597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.730878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.734586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:32.742375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.743967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:32.758993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:32.760099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:32.761144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 633/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:32.767007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:32.882532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020fddd-dc23-405a-8a9f-e1388579c2d7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:32.920857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:32.921565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:32.921862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:32.922133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:32.922973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:32.923286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:32.924719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:32.943931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 607/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:32.949670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.064329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9319e06-ac87-4322-b4f6-2591930b9685 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.107674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.108350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.108641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.108963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.176984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.183841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.185365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.302439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 634/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:33.308843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.340569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.423633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52069eb1-a67e-4176-a6ba-a70cb3347abd None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.469486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.470250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.470537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.470817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.482995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.484492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:33.505280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.506574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.507653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:33.507653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:33.507653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:33.507653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:33.507653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 608/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 00:17:33.515071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.634178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.641544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507eb50-0143-46cc-947a-44e9f608b32c None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.676457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.677201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.677450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.677729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.678459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:33.678784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.680148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.693391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 635/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:33.699046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:33.788557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:33.813485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c69b3-5be7-4989-ae05-3e2e1608739b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:33.853058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:33.854169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:33.854456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:33.854775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:33.862462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:33.863911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:33.877881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:33.879018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:33.880145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 609/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:33.885599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.004257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79eed87-a1de-4a83-8f42-f7bbf85a558d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.046272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.046983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.047277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.047562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.048363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:34.048724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.050334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a6def-f13c-425a-a1bf-a34806cd3e64 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.064617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 636/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:34.070998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:34.188584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e934c2-3087-4ae2-ae34-79e095e68af0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:34.224861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:34.225507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:34.225793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:34.226071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:34.226912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] RBAC: Adding query filter params (role.id=34a2aafa93ee4002bfed71c1f3acca43, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=2add2dc7d84c497a937723b3badcc881) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:34.227224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:34.228677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a2554-e81f-4f5c-8c5d-a6ebf9ff6434 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:34.235753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 610/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:34 2022] GET /identity/v3/role_assignments?user.id=2add2dc7d84c497a937723b3badcc881&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=34a2aafa93ee4002bfed71c1f3acca43 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:36.560951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658406c-ca9b-4f8a-bc21-48bc800c242f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.561532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658406c-ca9b-4f8a-bc21-48bc800c242f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.561805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658406c-ca9b-4f8a-bc21-48bc800c242f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.563667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 637/1247] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:36.570762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.571260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.571370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.576568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.591333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:36.592633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:36.669508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 611/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:36.692960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658406c-ca9b-4f8a-bc21-48bc800c242f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:36.714775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:36.819726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658406c-ca9b-4f8a-bc21-48bc800c242f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:36.863293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:36.864140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:36.864435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:36.864717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:36.875919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:36.884233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:36.885272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:36.886139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 638/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:36.892684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.004575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ba649-8f50-4891-951f-4202059726a1 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.038137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.043842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.044526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.044822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.045142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.045947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:37.046298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.047814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.056400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 612/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:37.062971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.145690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.170474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d4a02-213f-4800-9055-a57a7e79ee4b None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.209469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.210306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.210601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.210866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.229883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.232179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:37.255615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:37.256867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:37.257933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:37.257933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:37.257933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:37.257933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:37.257933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 639/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:37.263108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.374869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea0e7-7dac-4801-927c-8afed20eb541 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.410443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.411101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.411432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.411738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.412521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:37.412838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.414381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.427606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 613/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:37.433739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.566803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549e0ac-91ef-4fba-bd73-1c72484edec8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.596858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.605357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.607336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.607903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.608232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.617360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.619017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:37.638520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:37.639788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:37.640833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 640/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:37.646064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.707146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:37.757379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bb6bf-014b-40cd-9e4f-ba7f7ede540d None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.796101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.796873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.797193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.797518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:37.798359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:37.798710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:37.800219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:37.820777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 614/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:37.827005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:37.946761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05056-3752-4236-8489-33864983fb61 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:37.984824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:37.985421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:37.985711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:37.985993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.045832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.058858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.060431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.163895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 641/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 00:17:38.169977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.276501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbc804-63e3-485d-8949-a2a62eabf1c6 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.292074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.314814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.315629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.315915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.316192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.328227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.329733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:38.348535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:38.349724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:38.350826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:38.350826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:38.350826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:38.350826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:38.350826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 615/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:38.357466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.444496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.457785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccb74d-b3ec-46f7-a33a-5d0e68777022 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.492915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.493733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.493908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.494268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.495325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.495900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.499100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.515425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 642/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:38.521804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.636598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab1879-350c-4272-b2f2-6c641c609969 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.670472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.671253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.671534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.671813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.678320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.679755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:38.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:38.694279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:38.695317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 616/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:38.700893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.801713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc1a24-bdd5-4892-a683-c032ef81eddf None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.837164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.837819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.838107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.838388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.839173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.839486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.841937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab629-65dd-4e65-93db-eb633555b8ce None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:38.853809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 643/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:38.861796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:38.888244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:38.960959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1a57-13ea-456c-a4bf-6bda55087694 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:38.993291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:38.993935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:38.994218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:38.994491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:38.995314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=2add2dc7d84c497a937723b3badcc881) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:38.995624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:38.997093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69076644-c360-4126-afc8-c024a0f5b72f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:39.002329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 617/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:38 2022] GET /identity/v3/role_assignments?user.id=2add2dc7d84c497a937723b3badcc881&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.119811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6bab-74e8-4717-af47-1b6aa5308d42 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.120362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6bab-74e8-4717-af47-1b6aa5308d42 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.120362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6bab-74e8-4717-af47-1b6aa5308d42 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.122264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 644/1261] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:41.127307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.127727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.127881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.144856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:41.146071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:41.211242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 618/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 00:17:41.233602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6bab-74e8-4717-af47-1b6aa5308d42 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.244183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.332593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6bab-74e8-4717-af47-1b6aa5308d42 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.365687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.366468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.366755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.367031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.377881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:41.385436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:41.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:41.387851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 645/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:41.392866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.411225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.490964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194bf40-14bc-4d2d-b80c-676e03faddf6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.531807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.532550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.532962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.533345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.534200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.534568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.536221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.546974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 619/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.553245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.623853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.656648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c1341-fc65-4313-8ad7-6017b25f13e0 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.689063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.689849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.690131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.690405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.700625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.702082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:41.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:41.720337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:41.721326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:41.721326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:41.721326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:41.721326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:41.721326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 646/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:41.726147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.807211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:41.826956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25834d5-d1c9-4e61-bbc5-812db7448683 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:41.861172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:41.861830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:41.862120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:41.862401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:41.863129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:41.863442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:41.864804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:41.876303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 620/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:41.881906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:41.984326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997ca46-100f-4a83-b690-ad8f86bc0b8d None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.003300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.017631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.018451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.018741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.019017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.026530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.028552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:42.042612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:42.044285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:42.045522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 647/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:42.049622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.145584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23936805-6a83-4894-b168-24b1475896f0 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.179145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.179816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.180106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.180387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.181270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.181586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.183020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.191662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.200854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 621/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.206471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.303091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35721b-851d-4165-a9dd-90f7baed0ac8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.335597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.336404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.336693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.337006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.348809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.350283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:42.367127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:42.368275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:42.369385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:42.369385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:42.369385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:42.369385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:42.369385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 648/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:42.374436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.469667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b6a00-97a8-4496-84ad-222ce8a886e2 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.502833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.503494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.503779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.504058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.504817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.505121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.506746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.518809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 622/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:42.525266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.558333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.624442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d525d-809a-4b2a-b8d6-43ba490d1718 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.657159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.657943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.658228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.658507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.664861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.666306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:42.681239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:42.682310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:42.683433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 649/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:42.688707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.748775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.788954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2e22-6cb7-4f06-86c8-69aa6331e152 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.821601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.822259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.822548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.822828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.823615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:42.823926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.825452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c0ff0-6358-42f2-a13d-eb32b36bd042 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.837135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 623/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:17:42.843371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:42.936581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:42.945930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840493e-02e5-46f7-a2a6-2678203e962e None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:42.978998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:42.979876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:42.980171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:42.980450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] PATH_INFO: `/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/2add2dc7d84c497a937723b3badcc881/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:42.983412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] RBAC: Authorizing `identity:create_grant(project_id=f127d778cf9c45b898cca7a6089e15e6, user_id=2add2dc7d84c497a937723b3badcc881, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:42.984840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c2bfd-ae8e-4781-a7f6-db9501228f41 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:42.997638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 650/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Jun 16 00:17:42 2022] PUT /identity/v3/projects/f127d778cf9c45b898cca7a6089e15e6/users/2add2dc7d84c497a937723b3badcc881/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:17:45.125346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da4cd-7095-4a98-a43a-a591fb8de8eb None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.125877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da4cd-7095-4a98-a43a-a591fb8de8eb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.125877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da4cd-7095-4a98-a43a-a591fb8de8eb None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.127274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 624/1274] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:45.132469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.132859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.133074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.150136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:45.151321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:45.216242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 651/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:45.239367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da4cd-7095-4a98-a43a-a591fb8de8eb None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.337916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da4cd-7095-4a98-a43a-a591fb8de8eb None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.354737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.371693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.372471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.372759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.373069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.382874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 00:17:45.390462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.391573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:45.392625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 625/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:45.397911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.476034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.494765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24835d2-2220-465c-b272-d62dc67c51d3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.534556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.535385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.535677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.536026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.536726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:45.537068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.538572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.548109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 652/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:45.554393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.650545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c71cf-11e9-477f-b977-eef41f0db3f9 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.685225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.686010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.686298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.686581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.696717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.698179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:45.715650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:45.716745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:45.718136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:45.718136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:45.718136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:45.718136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:45.718136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 626/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:45.722637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.823229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f011-416f-49c9-95bb-bc08a30b3520 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:45.854799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:45.856050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:45.856333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:45.856608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:45.857358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:45.857671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:45.859033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:45.871165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 653/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:45.877583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:45.907738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:45.975229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c8a92-532c-437a-83b8-8e43daa5c45b None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.008332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.009173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.009456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.009732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.017139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.018586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 00:17:46.032691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:46.033804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:46.034822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 627/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:46.039774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.044788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.137410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840f17-5e3c-4fd0-8509-0efef41fe92a None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.170251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.170853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.171136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.171412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.172241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.172551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.173988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.190652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 654/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:46.196297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.283446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.292873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f528ef-d15d-4444-9162-c9fd990597c7 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.325443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.326081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.326364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.326642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.391158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.392585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.476917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.487747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 628/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 00:17:46.493520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.562277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.599415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada434d-84e5-4d95-989e-b142016b58d9 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.633280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.634075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.634362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.634642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.644921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.646363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 00:17:46.664274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:46.665340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:46.666301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 00:17:46.666301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 00:17:46.666301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 00:17:46.666301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:46.666301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 655/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:46.671907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.767625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37428751-6e31-4f81-953b-6e37c3674484 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.800413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.801096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.801379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.801654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.802372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:46.802682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.804049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.816018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 629/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:46.821850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:46.918750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61cdef-4e38-4a7e-b0f2-c42c1ec5e322 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:46.932106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:46.953466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:46.954256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:46.954545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:46.954826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:46.961346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:46.962778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 00:17:46.976364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:46.977391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: ERROR keystone.server.flask.application Jun 16 00:17:46.978565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 656/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:46.983735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.032569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:47.081329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6948b22-9be4-4a75-8072-c642d0952c58 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:47.112767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:47.113435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.113719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.113995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.114775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:47.115084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:47.116500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad23604-6cd2-47bd-a09a-ea926e28d088 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:47.128945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 630/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Jun 16 00:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:47.134963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:47.234822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8786fe-330a-47f5-9412-e28f324f05b8 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:47.267734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:47.268390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:47.268680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:47.268992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:47.269831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] RBAC: Adding query filter params (role.id=8e8886c4262d40709f5bdc535e1e89cb, scope.project.id=f127d778cf9c45b898cca7a6089e15e6, user.id=2add2dc7d84c497a937723b3badcc881) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:47.270154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:47.271585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a575525-a4f8-4864-9cb4-3555991b66f2 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:47.277729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 657/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Jun 16 00:17:47 2022] GET /identity/v3/role_assignments?user.id=2add2dc7d84c497a937723b3badcc881&scope.project.id=f127d778cf9c45b898cca7a6089e15e6&role.id=8e8886c4262d40709f5bdc535e1e89cb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:49.406057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650fb44-5f74-4e33-b74b-5c14ca834369 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.406603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650fb44-5f74-4e33-b74b-5c14ca834369 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.406603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650fb44-5f74-4e33-b74b-5c14ca834369 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.407982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 631/1288] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:49.413004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.413412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.413576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.430704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:49.431879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:49.460994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.498548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 658/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:49.524525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650fb44-5f74-4e33-b74b-5c14ca834369 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.627913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650fb44-5f74-4e33-b74b-5c14ca834369 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.638641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:49.661757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.662481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.662761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.663032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.663681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.665062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:49.676503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:49.677774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:49.678831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 632/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:49.686089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.784035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0119d-7f35-447b-95e4-375ced8fb39f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.816638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.817331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.817619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.817895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.818606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:49.818916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.820275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae648d-ad6c-4fc9-9f81-e1c6716b6974 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.825280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 659/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:49.830983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:49.928327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3f5b-b51b-4615-aade-a70b9def9335 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:49.960027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:49.960684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:49.961002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:49.961286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] PATH_INFO: `/v3/services` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:49.961991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:49.962308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:49.963673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1bdd4-35bc-4f61-a87f-880d03af9d08 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:49.968640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 633/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Jun 16 00:17:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:52.115967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61d35-56c7-4645-90f5-84a01162b4f3 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:52.116499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61d35-56c7-4645-90f5-84a01162b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:52.116499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e61d35-56c7-4645-90f5-84a01162b4f3 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:52.117922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 660/1293] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:52.123822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d7fb-e276-40b2-98c1-2ee0b42df68a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:52.124259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d7fb-e276-40b2-98c1-2ee0b42df68a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:52.124368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d7fb-e276-40b2-98c1-2ee0b42df68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:52.141849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4aa2d7fb-e276-40b2-98c1-2ee0b42df68a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:52.143019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4aa2d7fb-e276-40b2-98c1-2ee0b42df68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:52.173541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:52.224245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 634/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:52.248490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e61d35-56c7-4645-90f5-84a01162b4f3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:52.279310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:52.346329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e61d35-56c7-4645-90f5-84a01162b4f3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:52.379212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:52.379806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:52.380091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:52.380365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:52.381110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:52.382467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1818f-92a7-4090-bf24-416ba55a14a5 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:52.395161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 661/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Jun 16 00:17:52 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:17:54.524974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae128dd-02c2-404e-9ff4-8f5e62c3927a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.525508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae128dd-02c2-404e-9ff4-8f5e62c3927a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.525603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae128dd-02c2-404e-9ff4-8f5e62c3927a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.527030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 635/1296] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:54.532061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.532438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.532619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.549028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:54.550200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:54.619211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 662/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:54.641641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae128dd-02c2-404e-9ff4-8f5e62c3927a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.745180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae128dd-02c2-404e-9ff4-8f5e62c3927a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.778313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.779038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.779322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.779598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] PATH_INFO: `/v3/services/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.780244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.781639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:54.793068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:54.794284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:54.795343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 636/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:54.800245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.817981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:54.897897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96b12-410f-4096-82f9-cb38a5cd9bdf None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:54.930717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:54.931377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:54.931668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:54.931947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:54.932655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:54.933097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:54.934599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:54.939625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 663/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:54.945647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:54.976784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:55.042995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d89c8d-344e-4360-8394-400b61bc078f None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:55.075310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:55.075953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:55.076237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:55.076514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:55.077324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] RBAC: Adding query filter params (interface=public, service_id=ced5367ce7b641f49b81d31ca69a00b7, region_id=RegionOne) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:55.077639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:55.078993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738dc62-3bc4-4b1d-8072-7aa7b00789b0 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:55.083859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 637/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Jun 16 00:17:54 2022] GET /identity/v3/endpoints?service_id=ced5367ce7b641f49b81d31ca69a00b7&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:55.089511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:55.168834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:55.187228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c4565-df47-4901-89b4-db7525d11de3 None admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:55.220330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:55.221018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:55.221311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:55.221594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:55.222282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:55.223645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc9d8a-fef3-41aa-ba0b-6419e15843aa None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:55.230415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 664/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Jun 16 00:17:55 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:17:57.382464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a0b5-c6f1-47a7-8218-94b654f88e63 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.382996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a0b5-c6f1-47a7-8218-94b654f88e63 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.382996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a0b5-c6f1-47a7-8218-94b654f88e63 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.384396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 638/1302] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:17:57.389685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.390075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.390258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.407390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:17:57.408575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:17:57.476416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 665/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:17:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 00:17:57.500344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a0b5-c6f1-47a7-8218-94b654f88e63 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.594989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:57.601923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a0b5-c6f1-47a7-8218-94b654f88e63 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.634343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.635079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.635364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.635644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.636343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.637909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 00:17:57.649771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 00:17:57.650897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application with Lock( Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self._enter() Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application return self.creator() Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application created_value = creator( Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: ERROR keystone.server.flask.application Jun 16 00:17:57.652397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 639/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 00:17:57.657513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.732456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:17:57.754534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50bd83-d4fb-403d-a139-04c451be5fc9 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.788016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.788673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.788982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.789260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] PATH_INFO: `/v3/services` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.789958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] RBAC: Adding query filter params (name=placement) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:17:57.790262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.791615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bd78c-8c21-4b83-b5e3-f6537503c51d None admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:57.796811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 666/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Jun 16 00:17:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:17:57.802508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:17:57.898261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b0458-9c6c-4a80-bc72-ad896903c761 None admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:17:57.930700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:17:57.931346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:17:57.931634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:17:57.931911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:17:57.932644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:17:57.934109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2f02-2315-4aef-84df-f0707ca5afc2 None admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:17:57.951915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 640/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Jun 16 00:17:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:18:07.793829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=30 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:18:07.947247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:01.226576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600158-bdf0-48eb-920d-7a1804f4534d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:01.226576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600158-bdf0-48eb-920d-7a1804f4534d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:01.227365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600158-bdf0-48eb-920d-7a1804f4534d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:01.228451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 667/1307] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:01.234227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f53b-b2a2-490c-ac27-a0ffc55e5e86 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:01.234227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f53b-b2a2-490c-ac27-a0ffc55e5e86 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:01.234866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f53b-b2a2-490c-ac27-a0ffc55e5e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:01.358157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1477f53b-b2a2-490c-ac27-a0ffc55e5e86 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:01.359563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1477f53b-b2a2-490c-ac27-a0ffc55e5e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:01.506330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 641/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:01.559208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:01.559533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:01.559786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:01.561382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 668/1309] 10.222.0.23 () {54 vars in 964 bytes} [Thu Jun 16 00:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:01.567528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aff0-7693-4515-9c6b-eebd2a0a28e9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:01.567898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aff0-7693-4515-9c6b-eebd2a0a28e9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:01.568251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aff0-7693-4515-9c6b-eebd2a0a28e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:01.649376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0ff1aff0-7693-4515-9c6b-eebd2a0a28e9 None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:01.650712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0ff1aff0-7693-4515-9c6b-eebd2a0a28e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:01.673159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:01.744749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 642/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Jun 16 00:19:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:19:01.753844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:01.755705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:01.873272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d7810-9481-46d3-9900-d12e4499cc13 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:01.956623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:01.957411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:01.957705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:01.957982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:01.959618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:02.019523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:02.078761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:02.080198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12057c2-62f1-4d5e-b4ad-fcb7f635b803 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:02.105914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 669/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:19:04.588412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:04.588973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:04.589160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:04.590365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 643/1312] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:04.595786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65834c7-f759-436f-af7c-0a6cd86449ba None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:04.596226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65834c7-f759-436f-af7c-0a6cd86449ba None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:04.596427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65834c7-f759-436f-af7c-0a6cd86449ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:04.673915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e65834c7-f759-436f-af7c-0a6cd86449ba None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:04.675397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e65834c7-f759-436f-af7c-0a6cd86449ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:04.797792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 670/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:19:04.840309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:04.841823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:04.963500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd49cd-4ea5-4b88-86aa-2c9a59ed301d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:05.039344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:05.042536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:05.043188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:05.043858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:05.044525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:05.046602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:05.163665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:05.165159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa742ce-a2b9-4891-9687-61b96ba0e9c8 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:05.185352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 644/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:19:07.608334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc0f-86a3-47fc-8efa-af320328c656 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:07.608905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc0f-86a3-47fc-8efa-af320328c656 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:07.608905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc0f-86a3-47fc-8efa-af320328c656 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:07.610294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 671/1315] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:07.615320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f45-c2ae-4418-ace1-d733a9ddcbb6 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:07.615760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f45-c2ae-4418-ace1-d733a9ddcbb6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:07.616496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65f45-c2ae-4418-ace1-d733a9ddcbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:07.680958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fce65f45-c2ae-4418-ace1-d733a9ddcbb6 None None] MFA Rules not processed for user `5c4be08c17964e0f904d1c3c701380af`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:07.682162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fce65f45-c2ae-4418-ace1-d733a9ddcbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:07.732850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:07.777321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 645/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:19:07.831914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:07.832205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:07.832485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:07.833890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 672/1317] 10.222.0.23 () {54 vars in 964 bytes} [Thu Jun 16 00:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:07.840162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d08f-61cd-42e2-bd04-110f655a2722 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:07.840452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d08f-61cd-42e2-bd04-110f655a2722 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:07.840735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827d08f-61cd-42e2-bd04-110f655a2722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:07.912557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b827d08f-61cd-42e2-bd04-110f655a2722 None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:07.914375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b827d08f-61cd-42e2-bd04-110f655a2722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:07.988996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 646/1318] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Jun 16 00:19:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:07.998334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:07.999850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:08.009976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:08.104795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068c868-afa5-4b6c-9c8d-be12a23871be None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:08.176309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:08.176907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:08.177190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:08.177466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:08.179115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:08.239297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:08.286135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:08.287568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c080ee-5f77-45b3-b372-3bb1c13b5e38 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:08.305863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 673/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:11.381045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:11.382222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:11.382222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:11.383501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 647/1320] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:11.388591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387aab-3809-40f5-9512-7054e28697ff None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:11.389047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387aab-3809-40f5-9512-7054e28697ff None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:11.389189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f387aab-3809-40f5-9512-7054e28697ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:11.452820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7f387aab-3809-40f5-9512-7054e28697ff None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:11.453990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7f387aab-3809-40f5-9512-7054e28697ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:11.552344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 674/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:11.591607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:11.593246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:11.645584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:11.699808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939e4e9-fffc-4fc9-9b6d-91171a057473 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:11.774101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:11.774755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:11.775046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:11.775332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:11.777017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:11.876838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:11.891260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:11.892730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2fb6d-e571-41a3-9be7-cad1b3978ac8 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:11.910629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 648/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:14.802259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:14.802807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:14.802934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:14.804359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 675/1323] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:14.809589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49b15-fffe-4228-8de8-15a6858c40aa None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:14.810045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49b15-fffe-4228-8de8-15a6858c40aa None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:14.810146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f49b15-fffe-4228-8de8-15a6858c40aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:14.882711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-94f49b15-fffe-4228-8de8-15a6858c40aa None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:14.883958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-94f49b15-fffe-4228-8de8-15a6858c40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:14.992748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 649/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:15.030431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:15.032017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:15.051436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:15.139018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe9ea-31e1-4424-a31d-824672a3dd1a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:15.222906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:15.223565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:15.223856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:15.224137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:15.226043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:15.299232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:15.338298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:15.339732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c61da-8a64-403d-be7b-4d8726e01ef9 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:15.357997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 676/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:17.522112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:17.522786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:17.522786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:17.524087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 650/1326] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:17.529377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4f2f-939f-4b5e-bcf4-3e767b4482dc None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:17.529779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4f2f-939f-4b5e-bcf4-3e767b4482dc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:17.529951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab4f2f-939f-4b5e-bcf4-3e767b4482dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:17.594707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-baab4f2f-939f-4b5e-bcf4-3e767b4482dc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:17.595924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-baab4f2f-939f-4b5e-bcf4-3e767b4482dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:17.690165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 677/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:17.726231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:17.727916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:17.789714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:17.841010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c0550-fc7c-49b8-b252-bf7bbee05132 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:17.916705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:17.917402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:17.917693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:17.917974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:17.919616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:18.042683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:18.045172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:18.046658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84ea19-51db-4797-bf4b-261a76fac863 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:18.065830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 651/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:19:20.250156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:20.250700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:20.250700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:20.252085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 678/1329] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:20.257183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e65-ee0a-41dd-ac98-44fcdfca1725 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:20.257573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e65-ee0a-41dd-ac98-44fcdfca1725 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:20.257752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e65-ee0a-41dd-ac98-44fcdfca1725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:20.321238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-afe21e65-ee0a-41dd-ac98-44fcdfca1725 None None] MFA Rules not processed for user `5c4be08c17964e0f904d1c3c701380af`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:20.322438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-afe21e65-ee0a-41dd-ac98-44fcdfca1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:20.416707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 652/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:20.453786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:20.455383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:20.487126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:20.563753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980bf8d-ea5a-4f39-aa37-237a048a7651 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:20.660551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:20.661330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:20.661680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:20.662674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:20.664511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:20.744776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:20.776865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:20.778322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9b47a-17fb-4950-ac9d-197331a779ff service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:20.796402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 679/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:23.199379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:23.199875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:23.200002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:23.201367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 653/1332] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:23.206729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cd59c-843f-4104-960c-5d65c99bb276 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:23.207135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cd59c-843f-4104-960c-5d65c99bb276 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:23.207302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cd59c-843f-4104-960c-5d65c99bb276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:23.276507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-930cd59c-843f-4104-960c-5d65c99bb276 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:23.277735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-930cd59c-843f-4104-960c-5d65c99bb276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:23.381611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 680/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:23.417164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:23.418697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:23.487683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:23.523360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27e5f8-c069-4215-a348-0da22658574a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:23.594910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:23.595557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:23.595838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:23.596117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:23.597775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:23.702756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:23.704198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0db4f-6688-462f-98ed-66a38a34fde1 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:23.714823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:23.723570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 654/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:25.869653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:25.870208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:25.870312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:25.871622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 681/1335] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:25.876643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d81d0-1454-4f92-bbf6-b09129ff709e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:25.877110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d81d0-1454-4f92-bbf6-b09129ff709e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:25.877237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d81d0-1454-4f92-bbf6-b09129ff709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:25.948811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a08d81d0-1454-4f92-bbf6-b09129ff709e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:25.950058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a08d81d0-1454-4f92-bbf6-b09129ff709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:26.047215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 655/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:26.088359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:26.090020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:26.135093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:26.209286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86ad2d-e46b-497f-89ec-7be77363367c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:26.292489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:26.293180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:26.293468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:26.293748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:26.295438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:26.386628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:26.417742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:26.419191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de171ace-7208-4c1b-9932-74a80a9a7418 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:26.438690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 682/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:19:28.923799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:28.924379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:28.924379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:28.925802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 656/1338] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:28.931437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e184e-b62d-4c20-bf44-fef500055351 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:28.931858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e184e-b62d-4c20-bf44-fef500055351 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:28.932004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e184e-b62d-4c20-bf44-fef500055351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:28.995139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-917e184e-b62d-4c20-bf44-fef500055351 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:28.996259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-917e184e-b62d-4c20-bf44-fef500055351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:29.090815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 683/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:29.126037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:29.127575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:29.203632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:29.231335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0f6a-62c6-417b-9620-9f93a5493645 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:29.304056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:29.304627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:29.304938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:29.305217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:29.306866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:29.410751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:29.412771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17916699-6d8c-4050-bcdf-a0626aaaaa7a service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:29.430765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 657/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:33.563607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:33.564141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:33.564274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:33.565585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 684/1341] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:33.571223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbfb2d-faf4-4551-9316-53ea619797c0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:33.571662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbfb2d-faf4-4551-9316-53ea619797c0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:33.571879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbfb2d-faf4-4551-9316-53ea619797c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:33.578231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:33.641796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3adbfb2d-faf4-4551-9316-53ea619797c0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:33.642923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3adbfb2d-faf4-4551-9316-53ea619797c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:33.750639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 658/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:33.789822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:33.791890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:33.843512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:33.890538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3df4b-4998-4891-aa9d-06455a9845ee None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:33.894151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:33.895056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:33.895341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:33.896797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:33.905934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:33.907367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25756bf0-da56-4ab8-b81e-044b624b26c5 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:33.917304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 685/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:36.725673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:36.726208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:36.726373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:36.727832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 659/1344] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:36.733758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0d6fb-a147-4791-9f6b-2a56964bf644 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:36.734275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0d6fb-a147-4791-9f6b-2a56964bf644 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:36.734492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0d6fb-a147-4791-9f6b-2a56964bf644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:36.765924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-acd0d6fb-a147-4791-9f6b-2a56964bf644 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:36.767163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-acd0d6fb-a147-4791-9f6b-2a56964bf644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:36.859966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 686/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:36.900750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:36.902753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:36.916159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7258bd0-ff76-4032-ac40-ff54592bf24f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:36.919844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:36.920492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:36.920774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:36.921094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:36.932628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:36.934125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddc65b-1fed-4302-b1fd-6fecf6356e5f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:36.953114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 660/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:41.474000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76db5-df6c-42b0-9a09-84258d8a9636 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:41.474597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76db5-df6c-42b0-9a09-84258d8a9636 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:41.474893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d76db5-df6c-42b0-9a09-84258d8a9636 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:41.476861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 687/1347] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:41.483113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d89b-e5e2-46b2-a550-6f8e095650a6 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:41.483646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d89b-e5e2-46b2-a550-6f8e095650a6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:41.483741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8d89b-e5e2-46b2-a550-6f8e095650a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:41.502111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eed8d89b-e5e2-46b2-a550-6f8e095650a6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:41.503344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eed8d89b-e5e2-46b2-a550-6f8e095650a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:41.513132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 661/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:19:41.552335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76db5-df6c-42b0-9a09-84258d8a9636 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:41.563332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d76db5-df6c-42b0-9a09-84258d8a9636 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:41.567570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:41.568354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:41.568725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:41.569104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:41.580688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:41.582173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066f24-e9ad-4b65-a6f9-0e6eacae7be1 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:41.592699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 688/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:46.642058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b913-e94a-4c32-9ed0-5e2334e712e5 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:46.642058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b913-e94a-4c32-9ed0-5e2334e712e5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:46.642058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80b913-e94a-4c32-9ed0-5e2334e712e5 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:46.642058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 662/1350] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:46.650769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa479b8-b46f-4aab-b9a8-d9b2a54ab710 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:46.651266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa479b8-b46f-4aab-b9a8-d9b2a54ab710 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:46.651354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa479b8-b46f-4aab-b9a8-d9b2a54ab710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:46.746292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9fa479b8-b46f-4aab-b9a8-d9b2a54ab710 None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:46.747496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9fa479b8-b46f-4aab-b9a8-d9b2a54ab710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:46.776563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:46.829446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 689/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:19:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:51.116052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877cab-2d1a-4f02-8f8a-b292378415be None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:51.116691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877cab-2d1a-4f02-8f8a-b292378415be None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:51.116942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12877cab-2d1a-4f02-8f8a-b292378415be None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:51.118643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 663/1352] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:51.124609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce097b-f712-461a-97e8-ef9cde3b325b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:51.125070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce097b-f712-461a-97e8-ef9cde3b325b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:51.125404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce097b-f712-461a-97e8-ef9cde3b325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:51.146967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-acce097b-f712-461a-97e8-ef9cde3b325b None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:51.148188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-acce097b-f712-461a-97e8-ef9cde3b325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:51.162580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 690/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:19:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:51.496700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=72 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:19:52.714152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3401-06b8-4060-8ed9-251201a3ded0 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:52.714784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3401-06b8-4060-8ed9-251201a3ded0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:52.714784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee3401-06b8-4060-8ed9-251201a3ded0 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:52.716113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 664/1354] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:52.721596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809900cc-ebf4-41ca-93c8-c9f535abf292 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:52.723084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809900cc-ebf4-41ca-93c8-c9f535abf292 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:52.723702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809900cc-ebf4-41ca-93c8-c9f535abf292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:52.742813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-809900cc-ebf4-41ca-93c8-c9f535abf292 None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:52.744016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-809900cc-ebf4-41ca-93c8-c9f535abf292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:52.752549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 691/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:19:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:56.485667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce385f4-d586-4243-b076-50ffd8aa7eae None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:56.486174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce385f4-d586-4243-b076-50ffd8aa7eae None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:56.486174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce385f4-d586-4243-b076-50ffd8aa7eae None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:56.487500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 665/1356] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:56.493673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c564c-67c7-4049-94a7-4dbc1737c1ef None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:56.494092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c564c-67c7-4049-94a7-4dbc1737c1ef None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:56.494248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c564c-67c7-4049-94a7-4dbc1737c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:56.512426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a67c564c-67c7-4049-94a7-4dbc1737c1ef None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:56.513662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a67c564c-67c7-4049-94a7-4dbc1737c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:56.523332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 692/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:56.582215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9566ff-d5c5-4ff7-b669-0af324b2b505 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:56.582510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9566ff-d5c5-4ff7-b669-0af324b2b505 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:56.582781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9566ff-d5c5-4ff7-b669-0af324b2b505 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:56.584364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 666/1358] 10.222.0.23 () {54 vars in 961 bytes} [Thu Jun 16 00:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:56.592685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de9d3-1fbf-4730-be94-bab45fca29dc None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:56.593120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de9d3-1fbf-4730-be94-bab45fca29dc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:56.593626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42de9d3-1fbf-4730-be94-bab45fca29dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:56.649673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f42de9d3-1fbf-4730-be94-bab45fca29dc None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:56.651241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f42de9d3-1fbf-4730-be94-bab45fca29dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:56.757127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 693/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Jun 16 00:19:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:19:56.763138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9566ff-d5c5-4ff7-b669-0af324b2b505 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:56.780474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9566ff-d5c5-4ff7-b669-0af324b2b505 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:56.787772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:56.789310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:56.789896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:56.790339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:56.803189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:56.805192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb247-de0e-48fc-8e2f-65b4c4601d37 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:56.819710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 667/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:19:59.230637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502a6f-b9e8-4553-b938-0d0ad6e5e1e9 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:59.231236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502a6f-b9e8-4553-b938-0d0ad6e5e1e9 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:59.231236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15502a6f-b9e8-4553-b938-0d0ad6e5e1e9 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:59.232949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 694/1361] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:19:59.238894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d8d0-8a32-4aa5-9d98-5191467270d2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:59.239451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d8d0-8a32-4aa5-9d98-5191467270d2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:59.239451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d8d0-8a32-4aa5-9d98-5191467270d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:59.259444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-87e7d8d0-8a32-4aa5-9d98-5191467270d2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:19:59.260622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-87e7d8d0-8a32-4aa5-9d98-5191467270d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:19:59.269934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 668/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:19:59.316146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15502a6f-b9e8-4553-b938-0d0ad6e5e1e9 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:19:59.330738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15502a6f-b9e8-4553-b938-0d0ad6e5e1e9 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:19:59.339303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:19:59.340275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:19:59.340780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:19:59.341328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:19:59.353686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:19:59.355324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4bab3-6f49-41bc-affd-b393eafbf14c service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:19:59.366658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 695/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:01.833613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bae5-5852-4b5d-b17c-63d3cd2cd70a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:01.834124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bae5-5852-4b5d-b17c-63d3cd2cd70a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:01.834124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51bae5-5852-4b5d-b17c-63d3cd2cd70a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:01.835516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 669/1364] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:20:01.840851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ab364-363e-436b-ab90-5fc063164a9b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:01.841246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ab364-363e-436b-ab90-5fc063164a9b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:01.841448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ab364-363e-436b-ab90-5fc063164a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:01.859224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-560ab364-363e-436b-ab90-5fc063164a9b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:01.860426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-560ab364-363e-436b-ab90-5fc063164a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:01.868860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 696/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:01.916391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aff52a-e654-4571-b171-2afada7c580c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:01.916713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aff52a-e654-4571-b171-2afada7c580c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:01.917046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aff52a-e654-4571-b171-2afada7c580c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:01.918448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 670/1366] 10.222.0.23 () {54 vars in 961 bytes} [Thu Jun 16 00:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:01.923895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f017a69-b1fb-4bb7-a537-b997864028cb None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:01.924279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f017a69-b1fb-4bb7-a537-b997864028cb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:01.924652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f017a69-b1fb-4bb7-a537-b997864028cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:01.943590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9f017a69-b1fb-4bb7-a537-b997864028cb None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:01.944777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9f017a69-b1fb-4bb7-a537-b997864028cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:01.954130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 697/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Jun 16 00:20:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:01.961776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aff52a-e654-4571-b171-2afada7c580c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:01.983480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aff52a-e654-4571-b171-2afada7c580c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:01.990207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:01.990860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:01.991146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:01.991427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:02.000132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:02.001817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651910bd-238c-4e53-91c2-2ab092d82635 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:02.012679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 671/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:04.266728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e76c5-c180-4033-ae02-b1bbddf11abc None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:04.267283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e76c5-c180-4033-ae02-b1bbddf11abc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:04.267283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e76c5-c180-4033-ae02-b1bbddf11abc None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:04.269447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 698/1369] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:20:04.274651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1568-3377-405e-a63a-103f870fafea None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:04.275078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1568-3377-405e-a63a-103f870fafea None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:04.275212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c1568-3377-405e-a63a-103f870fafea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:04.294000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b80c1568-3377-405e-a63a-103f870fafea None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:04.295430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b80c1568-3377-405e-a63a-103f870fafea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:04.304800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 672/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:04.351261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e76c5-c180-4033-ae02-b1bbddf11abc None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:04.366425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e76c5-c180-4033-ae02-b1bbddf11abc None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:04.372422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:04.373406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:04.373937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:04.374435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:04.383976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:04.386243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9d20-749b-442a-867c-b58ad3043a83 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:04.397638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 699/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:04.796962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dab5f-7b08-41b2-8b46-99aeb6302843 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:04.797259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dab5f-7b08-41b2-8b46-99aeb6302843 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:04.797744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dab5f-7b08-41b2-8b46-99aeb6302843 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:04.799111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 673/1372] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:04.806074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111a37-436d-4294-807d-c3a30eeaa277 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:04.806445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111a37-436d-4294-807d-c3a30eeaa277 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:04.806803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111a37-436d-4294-807d-c3a30eeaa277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:04.825572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f5111a37-436d-4294-807d-c3a30eeaa277 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:04.826766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f5111a37-436d-4294-807d-c3a30eeaa277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:04.835928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 700/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:05.148830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1674-b74c-458f-bf1d-fa72f20c4237 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:05.148830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1674-b74c-458f-bf1d-fa72f20c4237 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:05.148830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1674-b74c-458f-bf1d-fa72f20c4237 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:05.148830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 674/1374] 10.222.0.23 () {54 vars in 963 bytes} [Thu Jun 16 00:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:05.153731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290effe-c791-4e68-a15e-3f3d2a29d233 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:05.154092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290effe-c791-4e68-a15e-3f3d2a29d233 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:05.154442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290effe-c791-4e68-a15e-3f3d2a29d233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:05.163367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:05.217946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a290effe-c791-4e68-a15e-3f3d2a29d233 None None] MFA Rules not processed for user `8863fd88c1a2416c9fdb0148db03f6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:05.219331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a290effe-c791-4e68-a15e-3f3d2a29d233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:05.287098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 701/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Jun 16 00:20:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:20:05.294380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1674-b74c-458f-bf1d-fa72f20c4237 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:05.306121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1674-b74c-458f-bf1d-fa72f20c4237 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:05.309601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:05.310241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:05.310525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:05.310989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:05.321260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:05.322697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdb363-d311-4770-9769-15e54dcae534 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:05.333547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 675/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:06.593466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff591-c5d6-412d-b04c-1afcf87d1b57 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:06.594104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff591-c5d6-412d-b04c-1afcf87d1b57 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:06.594104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff591-c5d6-412d-b04c-1afcf87d1b57 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:06.595423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 702/1377] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:06.601794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58f77d-5588-4a94-9097-70603bc04c78 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:06.602208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58f77d-5588-4a94-9097-70603bc04c78 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:06.602497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58f77d-5588-4a94-9097-70603bc04c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:06.619910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3f58f77d-5588-4a94-9097-70603bc04c78 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:06.621135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3f58f77d-5588-4a94-9097-70603bc04c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:06.629213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 676/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:06.673385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ff591-c5d6-412d-b04c-1afcf87d1b57 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:06.688238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ff591-c5d6-412d-b04c-1afcf87d1b57 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:06.693659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:06.694295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:06.694580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:06.694858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:06.703063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:06.707252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d4bb1-4885-4ed6-9c2b-8653743d1ee5 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:06.715334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 703/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:08.949511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239aac-d181-41ff-af73-6dcd8a144b1f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:08.950032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239aac-d181-41ff-af73-6dcd8a144b1f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:08.950032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239aac-d181-41ff-af73-6dcd8a144b1f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:08.951330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 677/1380] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:08.956695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9b5e-d2ce-4f9c-bde3-3018297a9d1a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:08.957164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9b5e-d2ce-4f9c-bde3-3018297a9d1a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:08.957288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f9b5e-d2ce-4f9c-bde3-3018297a9d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:08.976124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7e6f9b5e-d2ce-4f9c-bde3-3018297a9d1a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:08.977487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7e6f9b5e-d2ce-4f9c-bde3-3018297a9d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:08.985487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 704/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:09.030023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb239aac-d181-41ff-af73-6dcd8a144b1f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:09.046316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb239aac-d181-41ff-af73-6dcd8a144b1f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:09.052501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:09.053212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:09.053503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:09.053787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:09.062662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:09.064251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c369f-a338-4ff7-94d3-7bcb1a4969ed service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:09.074651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 678/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:11.262554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae436d0b-e557-44a5-8b9e-4c71bc638e47 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:11.263147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae436d0b-e557-44a5-8b9e-4c71bc638e47 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:11.263147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae436d0b-e557-44a5-8b9e-4c71bc638e47 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:11.264478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 705/1383] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:20:11.269993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1ca1f-9113-4786-8296-80c51b4cad1f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:11.270424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1ca1f-9113-4786-8296-80c51b4cad1f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:11.270684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1ca1f-9113-4786-8296-80c51b4cad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:11.291352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-37b1ca1f-9113-4786-8296-80c51b4cad1f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:11.292603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-37b1ca1f-9113-4786-8296-80c51b4cad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:11.301196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 679/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:20:11.342761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae436d0b-e557-44a5-8b9e-4c71bc638e47 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:11.357480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae436d0b-e557-44a5-8b9e-4c71bc638e47 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:11.362959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:11.364030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:11.364418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:11.364685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:11.373908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:11.375476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164869f-1517-4f19-9b34-bdf6ce6dbb6c service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:11.386107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 706/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:13.590444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd426ef-0d10-46de-900a-b09c18a1c2da None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.591021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd426ef-0d10-46de-900a-b09c18a1c2da None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.591021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd426ef-0d10-46de-900a-b09c18a1c2da None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.592403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 680/1386] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:20:13.599456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29e45-3cee-4030-a4f8-e421336a3048 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.599924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29e45-3cee-4030-a4f8-e421336a3048 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.600374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29e45-3cee-4030-a4f8-e421336a3048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.627004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-adc29e45-3cee-4030-a4f8-e421336a3048 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:13.629120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-adc29e45-3cee-4030-a4f8-e421336a3048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:13.641099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 707/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:13.687164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd426ef-0d10-46de-900a-b09c18a1c2da None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:13.709573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd426ef-0d10-46de-900a-b09c18a1c2da None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:13.717960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:13.718957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:13.719458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:13.719877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:13.734110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:13.735738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ae323-d4a3-4ae5-93a3-0bfa03e8be38 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:13.745973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 681/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:16.422734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f1bd5-c09e-4c9a-be52-7fe98e88cf59 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.423481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f1bd5-c09e-4c9a-be52-7fe98e88cf59 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.423481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f1bd5-c09e-4c9a-be52-7fe98e88cf59 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.424942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 708/1389] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:16.430255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9848f5-c865-4138-bf54-79d8cc541485 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.430811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9848f5-c865-4138-bf54-79d8cc541485 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.430811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9848f5-c865-4138-bf54-79d8cc541485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.448793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fe9848f5-c865-4138-bf54-79d8cc541485 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:16.450005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fe9848f5-c865-4138-bf54-79d8cc541485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:16.458523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 682/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:16.503825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f1bd5-c09e-4c9a-be52-7fe98e88cf59 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:16.523729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f1bd5-c09e-4c9a-be52-7fe98e88cf59 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:16.530593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:16.531396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:16.531749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:16.532028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:16.543286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:16.545371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d1f2-f46e-47ca-8c84-aa21b65e9641 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:16.558085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 709/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:18.762395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984e876-6595-488b-8d0e-a252e263f3a8 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:18.762926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984e876-6595-488b-8d0e-a252e263f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:18.763021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984e876-6595-488b-8d0e-a252e263f3a8 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:18.764340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 683/1392] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:18.770632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e14de-fff2-407d-a3b5-571638751d1b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:18.771079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e14de-fff2-407d-a3b5-571638751d1b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:18.771171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e14de-fff2-407d-a3b5-571638751d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:18.788860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e80e14de-fff2-407d-a3b5-571638751d1b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:18.790131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e80e14de-fff2-407d-a3b5-571638751d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:18.797875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 710/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:18.842635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984e876-6595-488b-8d0e-a252e263f3a8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:18.859089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984e876-6595-488b-8d0e-a252e263f3a8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:18.866579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:18.867230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:18.867513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:18.867798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:18.876372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:18.878792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7e500-8843-402e-b4a6-d5ff1f09c9f7 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:18.888235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 684/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:20.643084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ddc05-11f0-45b6-b4a7-e121f2522c43 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.643914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ddc05-11f0-45b6-b4a7-e121f2522c43 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.643914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ddc05-11f0-45b6-b4a7-e121f2522c43 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.645299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 711/1395] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:20.650653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdb748-dff5-4c1b-9fa4-1b89df7964b1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.651068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdb748-dff5-4c1b-9fa4-1b89df7964b1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.651252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdb748-dff5-4c1b-9fa4-1b89df7964b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.668912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-09cdb748-dff5-4c1b-9fa4-1b89df7964b1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:20.670114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-09cdb748-dff5-4c1b-9fa4-1b89df7964b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:20.686109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 685/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:20.742777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a23b-65fa-4cdc-bac3-aeb3af6b9d51 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.743142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a23b-65fa-4cdc-bac3-aeb3af6b9d51 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.743495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662a23b-65fa-4cdc-bac3-aeb3af6b9d51 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.745148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 712/1397] 10.222.0.23 () {54 vars in 962 bytes} [Thu Jun 16 00:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:20.750206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372db8ae-07dd-41b0-9493-000bbe929b51 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.750507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372db8ae-07dd-41b0-9493-000bbe929b51 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.750788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372db8ae-07dd-41b0-9493-000bbe929b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.800484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-372db8ae-07dd-41b0-9493-000bbe929b51 None None] MFA Rules not processed for user `ea2357b286df4e828b9f5024d71544ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:20.801743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-372db8ae-07dd-41b0-9493-000bbe929b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:20.861983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 686/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Jun 16 00:20:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:20.869568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662a23b-65fa-4cdc-bac3-aeb3af6b9d51 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:20.882089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662a23b-65fa-4cdc-bac3-aeb3af6b9d51 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:20.886032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:20.886915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:20.887206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:20.887484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:20.899018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:20.900404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54993f9b-b8ca-4bb8-affc-dbe6c9169801 service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:20.913353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 713/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:21.060032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d58aea-d1f9-4c1f-91cf-d52fcf659027 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.060398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d58aea-d1f9-4c1f-91cf-d52fcf659027 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.060709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d58aea-d1f9-4c1f-91cf-d52fcf659027 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.062160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 687/1400] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:21.068009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65bca-6ab1-44d7-85c0-9ce7da1ff38b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.068325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65bca-6ab1-44d7-85c0-9ce7da1ff38b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.068586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65bca-6ab1-44d7-85c0-9ce7da1ff38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.086304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a7f65bca-6ab1-44d7-85c0-9ce7da1ff38b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:21.087497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a7f65bca-6ab1-44d7-85c0-9ce7da1ff38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:21.095316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 714/1401] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:21.141093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d58aea-d1f9-4c1f-91cf-d52fcf659027 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.156470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d58aea-d1f9-4c1f-91cf-d52fcf659027 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.162923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.163572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.163862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.164147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.172404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.174609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db4f9f-3ff7-4521-b944-05a8289081c9 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.184081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 688/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:21.647953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b1ab-db05-4f8a-a483-29518d7b872a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.648504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b1ab-db05-4f8a-a483-29518d7b872a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.648504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b1ab-db05-4f8a-a483-29518d7b872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.696845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-38b7b1ab-db05-4f8a-a483-29518d7b872a None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:21.698082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-38b7b1ab-db05-4f8a-a483-29518d7b872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:21.734199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.765883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 715/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:20:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:20:21.807430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc424f2-172d-42fe-896f-c835558d5c86 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.807934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc424f2-172d-42fe-896f-c835558d5c86 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.808190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc424f2-172d-42fe-896f-c835558d5c86 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.809741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 689/1404] 10.222.0.23 () {54 vars in 942 bytes} [Thu Jun 16 00:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:21.816764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f2728-761b-48ab-af5f-80e36991db68 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.817175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f2728-761b-48ab-af5f-80e36991db68 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.817527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f2728-761b-48ab-af5f-80e36991db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.865540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e85f2728-761b-48ab-af5f-80e36991db68 None None] MFA Rules not processed for user `e57e080ca56d4f28bb1692d147cf4caa`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:21.866761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e85f2728-761b-48ab-af5f-80e36991db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:21.942456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 716/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Thu Jun 16 00:20:21 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:20:21.951818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc424f2-172d-42fe-896f-c835558d5c86 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:21.959845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:21.967051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc424f2-172d-42fe-896f-c835558d5c86 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:21.972063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:21.972716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:21.973041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:21.973327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:21.981826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:21.983409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d88ba1a-8c5f-4bfb-afe7-1954fc61ae36 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:21.992138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 690/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:20:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:20:23.386532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce310c-7be0-46f8-9b4e-b4854f361316 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:23.387062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce310c-7be0-46f8-9b4e-b4854f361316 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:23.387254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce310c-7be0-46f8-9b4e-b4854f361316 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:23.388695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 717/1407] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:23.394905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350a952-2623-4cfd-93ab-f8d9015f1a63 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:23.395369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350a952-2623-4cfd-93ab-f8d9015f1a63 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:23.395475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350a952-2623-4cfd-93ab-f8d9015f1a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:23.415154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f350a952-2623-4cfd-93ab-f8d9015f1a63 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:23.416359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f350a952-2623-4cfd-93ab-f8d9015f1a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:23.426753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 691/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:23.471914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce310c-7be0-46f8-9b4e-b4854f361316 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:23.487067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce310c-7be0-46f8-9b4e-b4854f361316 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:23.494994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:23.495652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:23.495941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:23.496217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:23.506432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:23.508020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ea13c-9dcb-4985-9914-8a0a66898683 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:23.525525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 718/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:33.408918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=8 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:33.514241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=62 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:20:44.941696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32967-d3f4-4fb4-badc-9d7b3ff097eb None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:44.942325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32967-d3f4-4fb4-badc-9d7b3ff097eb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:44.942325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32967-d3f4-4fb4-badc-9d7b3ff097eb None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:44.943648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 692/1410] 10.222.0.23 () {54 vars in 918 bytes} [Thu Jun 16 00:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:44.948695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed106-4335-4a6f-9add-f7f9623f8b0b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:44.949210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed106-4335-4a6f-9add-f7f9623f8b0b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:44.949324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ed106-4335-4a6f-9add-f7f9623f8b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:44.967537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b07ed106-4335-4a6f-9add-f7f9623f8b0b None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:44.968752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b07ed106-4335-4a6f-9add-f7f9623f8b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:44.977921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 719/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Jun 16 00:20:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:44.992360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67f31-1be1-468c-ba74-f591d32fdf6e None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:44.992650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67f31-1be1-468c-ba74-f591d32fdf6e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:44.992974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb67f31-1be1-468c-ba74-f591d32fdf6e None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:44.994319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 693/1412] 10.222.0.23 () {54 vars in 918 bytes} [Thu Jun 16 00:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:44.998731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3919dd-5710-4ad8-a373-6ff3f592e7ff None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:44.999027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3919dd-5710-4ad8-a373-6ff3f592e7ff None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:44.999314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3919dd-5710-4ad8-a373-6ff3f592e7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:45.016945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3c3919dd-5710-4ad8-a373-6ff3f592e7ff None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:45.018140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3c3919dd-5710-4ad8-a373-6ff3f592e7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:45.027431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 720/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Jun 16 00:20:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:47.635080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f9b7-6371-43ab-91ba-74e8a97bf083 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:47.635623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f9b7-6371-43ab-91ba-74e8a97bf083 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:47.635623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580f9b7-6371-43ab-91ba-74e8a97bf083 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:47.637018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:47.642430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230359d-4bbd-4dfd-9b6b-45dbe664a4a7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:47.642876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230359d-4bbd-4dfd-9b6b-45dbe664a4a7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:47.642978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230359d-4bbd-4dfd-9b6b-45dbe664a4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:47.660855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f230359d-4bbd-4dfd-9b6b-45dbe664a4a7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:47.662056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f230359d-4bbd-4dfd-9b6b-45dbe664a4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:47.669631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:47.712290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580f9b7-6371-43ab-91ba-74e8a97bf083 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:47.727172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580f9b7-6371-43ab-91ba-74e8a97bf083 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:47.732384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:47.733054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:47.733339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:47.733611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:47.742296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:47.743881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785911b-9ba1-4afb-bf60-96667acb3c2c service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:47.753649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 695/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:48.718401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a14-d1a1-4540-9668-82b1a86c9239 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:48.718974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a14-d1a1-4540-9668-82b1a86c9239 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:48.718974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a14-d1a1-4540-9668-82b1a86c9239 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:48.721045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:48.726164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a7fcc-5b40-4e94-8b31-eb6fc1df1154 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:48.726614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a7fcc-5b40-4e94-8b31-eb6fc1df1154 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:48.726919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a7fcc-5b40-4e94-8b31-eb6fc1df1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:48.745354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f97a7fcc-5b40-4e94-8b31-eb6fc1df1154 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:48.746548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f97a7fcc-5b40-4e94-8b31-eb6fc1df1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:48.754397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:48.922904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e6a14-d1a1-4540-9668-82b1a86c9239 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:48.937955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e6a14-d1a1-4540-9668-82b1a86c9239 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:48.943327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:48.943959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:48.944244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:48.944522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:48.952402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:48.954001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3dff1-b5dd-4baf-a4bf-e797aa9bac4a service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:48.963063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 723/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:49.896355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceae0f-c11b-434a-aba9-56de813de34a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:49.896946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceae0f-c11b-434a-aba9-56de813de34a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:49.896946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceae0f-c11b-434a-aba9-56de813de34a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:49.898338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:49.903272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71949754-d268-48da-beeb-b2ecdc263d33 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:49.903686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71949754-d268-48da-beeb-b2ecdc263d33 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:49.903837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71949754-d268-48da-beeb-b2ecdc263d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:49.921561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-71949754-d268-48da-beeb-b2ecdc263d33 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:49.922742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-71949754-d268-48da-beeb-b2ecdc263d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:49.930904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:50.083153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceae0f-c11b-434a-aba9-56de813de34a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:50.097816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceae0f-c11b-434a-aba9-56de813de34a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:50.103187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:50.103827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:50.104121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:50.104398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:50.112424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:50.114035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4732f66-1340-45dd-9300-d73b2f066bc2 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:50.123325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 698/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:51.025393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae681-1403-484e-97f7-7a9fd6a828cb None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:51.025908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae681-1403-484e-97f7-7a9fd6a828cb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:51.025908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae681-1403-484e-97f7-7a9fd6a828cb None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:51.027246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:51.032231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5c13-f2e7-428d-a362-6299d4d4c5a0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:51.033589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5c13-f2e7-428d-a362-6299d4d4c5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:51.033589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e5c13-f2e7-428d-a362-6299d4d4c5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:51.051638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b42e5c13-f2e7-428d-a362-6299d4d4c5a0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:51.052822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b42e5c13-f2e7-428d-a362-6299d4d4c5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:51.060502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:51.212166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae681-1403-484e-97f7-7a9fd6a828cb None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:51.226843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae681-1403-484e-97f7-7a9fd6a828cb None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:51.232463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:51.233121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:51.233415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:51.233698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:51.242611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:51.244235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975f47f-7f82-4d6e-9106-efeee239928a service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:51.253824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 726/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:52.153223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7f9b9-113f-4608-b14c-6673105bf705 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:52.153763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7f9b9-113f-4608-b14c-6673105bf705 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:52.153763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c7f9b9-113f-4608-b14c-6673105bf705 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:52.155161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:52.160732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab2304-e664-4283-b72f-bc7a07842722 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:52.161154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab2304-e664-4283-b72f-bc7a07842722 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:52.161341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab2304-e664-4283-b72f-bc7a07842722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:52.179381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-85ab2304-e664-4283-b72f-bc7a07842722 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:52.180578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-85ab2304-e664-4283-b72f-bc7a07842722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:52.188285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:52.343617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7f9b9-113f-4608-b14c-6673105bf705 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:52.358176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c7f9b9-113f-4608-b14c-6673105bf705 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:52.363543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:52.364174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:52.364458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:52.364735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:52.372645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:52.374259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6e60e-135f-43f1-8763-b31a40b53e14 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:52.383599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 701/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:53.238067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389be9-4907-4e1b-97bb-b25469cd97d5 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:53.238502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389be9-4907-4e1b-97bb-b25469cd97d5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:53.238629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55389be9-4907-4e1b-97bb-b25469cd97d5 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:53.240014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:53.244999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd824-eea2-4d9a-af0f-2b527a94d1c7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:53.245413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd824-eea2-4d9a-af0f-2b527a94d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:53.245610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd824-eea2-4d9a-af0f-2b527a94d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:53.263242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-db4fd824-eea2-4d9a-af0f-2b527a94d1c7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:53.264439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-db4fd824-eea2-4d9a-af0f-2b527a94d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:53.272544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:53.428747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55389be9-4907-4e1b-97bb-b25469cd97d5 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:53.442956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55389be9-4907-4e1b-97bb-b25469cd97d5 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:53.448215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:53.448852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:53.449237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:53.449521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:53.457778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:53.460016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d192d-943d-451c-85a1-7f4ed8b358a8 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:53.474365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 729/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:54.326429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af94a0c-912f-414b-bfad-b06fe10a86d4 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:54.327000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af94a0c-912f-414b-bfad-b06fe10a86d4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:54.327000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af94a0c-912f-414b-bfad-b06fe10a86d4 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:54.328350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:54.333855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ae0a-abfe-436a-98aa-f22d1ab721a7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:54.334283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ae0a-abfe-436a-98aa-f22d1ab721a7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:54.334422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85ae0a-abfe-436a-98aa-f22d1ab721a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:54.352812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3d85ae0a-abfe-436a-98aa-f22d1ab721a7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:54.354031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3d85ae0a-abfe-436a-98aa-f22d1ab721a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:54.362259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:54.511629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af94a0c-912f-414b-bfad-b06fe10a86d4 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:54.527563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af94a0c-912f-414b-bfad-b06fe10a86d4 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:54.533487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:54.534123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:54.534408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:54.534688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:54.542640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:54.544240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc55eec-88d3-4cb5-a10a-de8cf3ade24f service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:54.555403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 704/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:55.424478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dfcf1-3e02-42a4-8d73-9e834711eac8 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:55.425057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dfcf1-3e02-42a4-8d73-9e834711eac8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:55.425057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dfcf1-3e02-42a4-8d73-9e834711eac8 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:55.426469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:55.431405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1a80-f2b6-4305-b493-1df0aa8696cc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:55.431813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1a80-f2b6-4305-b493-1df0aa8696cc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:55.431982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f1a80-f2b6-4305-b493-1df0aa8696cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:55.450233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ed9f1a80-f2b6-4305-b493-1df0aa8696cc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:55.451435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ed9f1a80-f2b6-4305-b493-1df0aa8696cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:55.459834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:55.611021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dfcf1-3e02-42a4-8d73-9e834711eac8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:55.625027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dfcf1-3e02-42a4-8d73-9e834711eac8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:55.631308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:55.631986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:55.632276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:55.632559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:55.641079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:55.643981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ec510-1d88-4313-8ba6-a2991f1f92cc service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:55.653347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 732/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:56.493291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b668a-7de8-4cc9-80ac-e5ead58b3e0c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:56.493865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b668a-7de8-4cc9-80ac-e5ead58b3e0c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:56.493865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b668a-7de8-4cc9-80ac-e5ead58b3e0c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:56.495219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:56.500032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ecc9c-f8fb-4e7b-87fc-bb012e82bf0d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:56.500477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ecc9c-f8fb-4e7b-87fc-bb012e82bf0d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:56.500576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ecc9c-f8fb-4e7b-87fc-bb012e82bf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:56.518553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-209ecc9c-f8fb-4e7b-87fc-bb012e82bf0d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:56.520006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-209ecc9c-f8fb-4e7b-87fc-bb012e82bf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:56.528282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:56.679360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b668a-7de8-4cc9-80ac-e5ead58b3e0c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:56.693266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b668a-7de8-4cc9-80ac-e5ead58b3e0c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:56.698559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:56.699193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:56.699476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:56.699760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:56.708086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:56.709694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e9c40-cf3c-4b3f-b317-43d2ef4bf680 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:56.719023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 707/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:20:58.836923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764d1b-80fc-41c0-a109-9b6c8a1dc80a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:58.837473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764d1b-80fc-41c0-a109-9b6c8a1dc80a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:58.837473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e764d1b-80fc-41c0-a109-9b6c8a1dc80a None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:58.838857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:20:58.843920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c92ab-e0af-4001-bce1-c56a63d418b0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:58.844312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c92ab-e0af-4001-bce1-c56a63d418b0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:58.844492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c92ab-e0af-4001-bce1-c56a63d418b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:58.862679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a78c92ab-e0af-4001-bce1-c56a63d418b0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:20:58.863862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a78c92ab-e0af-4001-bce1-c56a63d418b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:20:58.871520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:20:58.910878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e764d1b-80fc-41c0-a109-9b6c8a1dc80a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:20:58.921882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e764d1b-80fc-41c0-a109-9b6c8a1dc80a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:20:58.925480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:20:58.926115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:20:58.926399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:20:58.926677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:20:58.935414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:20:58.936828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aab378-8866-4160-abb9-fcdeab6b46ea service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:20:58.946213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 735/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:01.233680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd88a-f89f-42bb-a5fb-84434c14ad4d None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:01.234252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd88a-f89f-42bb-a5fb-84434c14ad4d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:01.234252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd88a-f89f-42bb-a5fb-84434c14ad4d None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:01.236211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:01.241794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ce8e-a7de-49c4-a825-241f57b3a3c6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:01.242216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ce8e-a7de-49c4-a825-241f57b3a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:01.242362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199ce8e-a7de-49c4-a825-241f57b3a3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:01.260201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0199ce8e-a7de-49c4-a825-241f57b3a3c6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:01.261417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0199ce8e-a7de-49c4-a825-241f57b3a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:01.269512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:21:01.307471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd88a-f89f-42bb-a5fb-84434c14ad4d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:01.318865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd88a-f89f-42bb-a5fb-84434c14ad4d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:01.322355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:01.322993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:01.323281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:01.323563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:01.341687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:01.344990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c90f2-38fa-4822-9338-e6ca5a8563e5 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:01.358325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 710/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:03.834120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb9e-77b1-492f-a589-f461093cde6f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:03.834951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb9e-77b1-492f-a589-f461093cde6f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:03.835112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb9e-77b1-492f-a589-f461093cde6f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:03.836559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:03.841760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129a94-1b58-4d32-920f-99151521146f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:03.842190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129a94-1b58-4d32-920f-99151521146f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:03.842365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71129a94-1b58-4d32-920f-99151521146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:03.860217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-71129a94-1b58-4d32-920f-99151521146f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:03.861898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-71129a94-1b58-4d32-920f-99151521146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:03.869553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:03.909363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60faeb9e-77b1-492f-a589-f461093cde6f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:03.919837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60faeb9e-77b1-492f-a589-f461093cde6f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:03.923427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:03.924066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:03.924357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:03.924636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:03.933348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:03.934762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f5671-89eb-46d9-8e56-8b857f9e5796 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:03.944496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 738/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:06.262500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9a7e6-eed2-4a24-9b56-d71736e880f7 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:06.263095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9a7e6-eed2-4a24-9b56-d71736e880f7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:06.263095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9a7e6-eed2-4a24-9b56-d71736e880f7 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:06.264478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:06.270192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb142351-ca10-48f3-a39f-e98c6faf4f66 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:06.270620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb142351-ca10-48f3-a39f-e98c6faf4f66 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:06.270782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb142351-ca10-48f3-a39f-e98c6faf4f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:06.289081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-eb142351-ca10-48f3-a39f-e98c6faf4f66 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:06.290279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-eb142351-ca10-48f3-a39f-e98c6faf4f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:06.298011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:06.337165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9a7e6-eed2-4a24-9b56-d71736e880f7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:06.347838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9a7e6-eed2-4a24-9b56-d71736e880f7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:06.351289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:06.351926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:06.352213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:06.352494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:06.361868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:06.363299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119cee2-b281-4986-81b7-9160dc12c964 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:06.372492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 713/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:08.720905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b83b-7188-4ee1-8b66-d1a34a0bcd70 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:08.721433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b83b-7188-4ee1-8b66-d1a34a0bcd70 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:08.721536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7b83b-7188-4ee1-8b66-d1a34a0bcd70 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:08.722850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:08.728225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da7a24-0ce2-4051-ab18-3f453313e102 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:08.728605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da7a24-0ce2-4051-ab18-3f453313e102 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:08.728793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da7a24-0ce2-4051-ab18-3f453313e102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:08.747403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a6da7a24-0ce2-4051-ab18-3f453313e102 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:08.748601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a6da7a24-0ce2-4051-ab18-3f453313e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:08.757763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:08.795553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7b83b-7188-4ee1-8b66-d1a34a0bcd70 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:08.806891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7b83b-7188-4ee1-8b66-d1a34a0bcd70 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:08.810334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:08.810972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:08.811257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:08.811533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:08.821452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:08.822915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825fce-0030-4f73-9dd5-61b2848a5b40 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:08.832505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 741/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:11.015561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d44a-9bde-42e5-bb1f-d109f0681ea6 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:11.016089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d44a-9bde-42e5-bb1f-d109f0681ea6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:11.016089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2d44a-9bde-42e5-bb1f-d109f0681ea6 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:11.017712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:11.023111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfb506-69a1-4f2e-9803-f70dbf3c4271 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:11.023519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfb506-69a1-4f2e-9803-f70dbf3c4271 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:11.023677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfb506-69a1-4f2e-9803-f70dbf3c4271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:11.041116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-27dfb506-69a1-4f2e-9803-f70dbf3c4271 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:11.042297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-27dfb506-69a1-4f2e-9803-f70dbf3c4271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:11.050652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:11.087935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2d44a-9bde-42e5-bb1f-d109f0681ea6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:11.098570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2d44a-9bde-42e5-bb1f-d109f0681ea6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:11.102132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:11.102771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:11.103059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:11.103342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:11.112743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:11.114208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694ebcb-ed66-4538-909f-b232d09d9f55 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:11.123452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 716/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:13.310105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc173220-56c9-4225-b298-8ae6cb0c8a2f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:13.310663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc173220-56c9-4225-b298-8ae6cb0c8a2f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:13.310663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc173220-56c9-4225-b298-8ae6cb0c8a2f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:13.312022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:13.317895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6699f8-8a81-41f5-b199-cca2fdfcbedf None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:13.318321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6699f8-8a81-41f5-b199-cca2fdfcbedf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:13.318466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6699f8-8a81-41f5-b199-cca2fdfcbedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:13.337269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-af6699f8-8a81-41f5-b199-cca2fdfcbedf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:13.338449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-af6699f8-8a81-41f5-b199-cca2fdfcbedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:13.346118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:13.383952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc173220-56c9-4225-b298-8ae6cb0c8a2f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:13.396214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc173220-56c9-4225-b298-8ae6cb0c8a2f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:13.400245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:13.400936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:13.401205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:13.401483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:13.411947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:13.413665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d116b-eb5e-45b8-95df-66610a812b09 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:13.423707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 744/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:15.578710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9eba-501b-49ee-b06e-ebdfdbc9355a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:15.579768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9eba-501b-49ee-b06e-ebdfdbc9355a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:15.579870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9eba-501b-49ee-b06e-ebdfdbc9355a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:15.581436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:15.586561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6341c9-ace1-4480-abe3-06ce8ad5de85 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:15.586953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6341c9-ace1-4480-abe3-06ce8ad5de85 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:15.587130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6341c9-ace1-4480-abe3-06ce8ad5de85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:15.593221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=79 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:21:15.607292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3a6341c9-ace1-4480-abe3-06ce8ad5de85 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:15.608499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3a6341c9-ace1-4480-abe3-06ce8ad5de85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:15.616609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:15.653999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e9eba-501b-49ee-b06e-ebdfdbc9355a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:15.659868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=73 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:21:15.666097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e9eba-501b-49ee-b06e-ebdfdbc9355a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:15.669555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:15.670189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:15.670472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:15.670748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:15.679633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:15.681111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08a5c7-4cf2-46ac-a029-bca2f953d314 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:15.690719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 719/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:17.882881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e05b-b946-4e3b-9951-92a4a785e32d None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:17.883408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e05b-b946-4e3b-9951-92a4a785e32d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:17.883500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e05b-b946-4e3b-9951-92a4a785e32d None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:17.884833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:17.889989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495393-e5db-4439-bbfb-5e44770f6063 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:17.890408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495393-e5db-4439-bbfb-5e44770f6063 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:17.890555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c495393-e5db-4439-bbfb-5e44770f6063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:17.909691 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1c495393-e5db-4439-bbfb-5e44770f6063 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:17.910872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1c495393-e5db-4439-bbfb-5e44770f6063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:17.918700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:17.955972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6e05b-b946-4e3b-9951-92a4a785e32d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:17.966488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6e05b-b946-4e3b-9951-92a4a785e32d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:17.971820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:17.972473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:17.972755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:17.973064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:17.982258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:17.983632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36d879-8bda-46c8-9176-a16152268259 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:17.993146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 747/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:20.155604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6afa-6bdf-4664-b47b-436c4946b38a None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:20.156170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6afa-6bdf-4664-b47b-436c4946b38a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:20.156170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb6afa-6bdf-4664-b47b-436c4946b38a None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:20.157690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:20.163353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819595fe-7872-403e-9518-d3949e5141c6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:20.163782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819595fe-7872-403e-9518-d3949e5141c6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:20.163917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819595fe-7872-403e-9518-d3949e5141c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:20.181847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-819595fe-7872-403e-9518-d3949e5141c6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:20.183039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-819595fe-7872-403e-9518-d3949e5141c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:20.191289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:20.227055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb6afa-6bdf-4664-b47b-436c4946b38a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:20.238068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb6afa-6bdf-4664-b47b-436c4946b38a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:20.241596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:20.242244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:20.242535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:20.242820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:20.252211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:20.253960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8478a-81fe-4c50-88c2-99886b81427d service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:20.263265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 722/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:30.176189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=11 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:21:30.259130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=14 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:21:40.391268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c21b-426c-4f3b-b323-703ad18ec5bd None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:40.391834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c21b-426c-4f3b-b323-703ad18ec5bd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:40.392349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509c21b-426c-4f3b-b323-703ad18ec5bd None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:40.393786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 749/1471] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:40.399314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47e67-bf9f-4afc-9ea3-de653f4fbc84 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:40.399759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47e67-bf9f-4afc-9ea3-de653f4fbc84 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:40.399867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47e67-bf9f-4afc-9ea3-de653f4fbc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:40.417527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e6c47e67-bf9f-4afc-9ea3-de653f4fbc84 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:40.418708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e6c47e67-bf9f-4afc-9ea3-de653f4fbc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:40.426928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 723/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:40.474734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18b70-553e-4481-aaa2-78b26790b61f None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:40.475031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18b70-553e-4481-aaa2-78b26790b61f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:40.475315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc18b70-553e-4481-aaa2-78b26790b61f None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:40.476678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 750/1473] 10.222.0.23 () {54 vars in 962 bytes} [Thu Jun 16 00:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:40.481441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897741e4-c2ae-413e-9e17-c96a5d844e4d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:40.481759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897741e4-c2ae-413e-9e17-c96a5d844e4d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:40.482076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897741e4-c2ae-413e-9e17-c96a5d844e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:40.500441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-897741e4-c2ae-413e-9e17-c96a5d844e4d None None] MFA Rules not processed for user `ea2357b286df4e828b9f5024d71544ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:40.501801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-897741e4-c2ae-413e-9e17-c96a5d844e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:40.508195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 724/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Jun 16 00:21:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:40.515595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc18b70-553e-4481-aaa2-78b26790b61f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:40.527041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc18b70-553e-4481-aaa2-78b26790b61f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:40.530504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:40.531136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:40.531418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:40.531691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:40.540947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:40.542359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39595da-cfc3-4b3e-b65a-8d7509de7427 service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:40.552228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 751/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:42.760898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37305ee-1ecf-4015-b378-0ca7d7a62f49 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:42.761399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37305ee-1ecf-4015-b378-0ca7d7a62f49 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:42.761548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37305ee-1ecf-4015-b378-0ca7d7a62f49 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:42.762972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 725/1476] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:42.768427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93ddb-6589-4d75-a7c2-0a7ab06483ac None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:42.768866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93ddb-6589-4d75-a7c2-0a7ab06483ac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:42.769016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93ddb-6589-4d75-a7c2-0a7ab06483ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:42.787440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c4c93ddb-6589-4d75-a7c2-0a7ab06483ac None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:42.788643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c4c93ddb-6589-4d75-a7c2-0a7ab06483ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:42.796405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 752/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:42.820611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37305ee-1ecf-4015-b378-0ca7d7a62f49 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:42.833143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37305ee-1ecf-4015-b378-0ca7d7a62f49 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:42.837599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:42.838246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:42.838533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:42.838812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:42.839548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:42.841339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:21:42.841339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:21:42.841339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:21:42.842479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be283-3d53-4395-b364-a8bad632e149 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:42.866164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 726/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Jun 16 00:21:42 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:21:45.060660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca9fe-c96f-49f2-9d7c-c2b7d2af673b None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:45.061234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca9fe-c96f-49f2-9d7c-c2b7d2af673b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:45.061234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ca9fe-c96f-49f2-9d7c-c2b7d2af673b None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:45.062579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 753/1479] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:45.067715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab54fc-9d18-442a-86a1-5dd30712f878 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:45.068097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab54fc-9d18-442a-86a1-5dd30712f878 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:45.068281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab54fc-9d18-442a-86a1-5dd30712f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:45.085528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ddab54fc-9d18-442a-86a1-5dd30712f878 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:45.086811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ddab54fc-9d18-442a-86a1-5dd30712f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:45.094528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 727/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:45.138300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ca9fe-c96f-49f2-9d7c-c2b7d2af673b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:45.153724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ca9fe-c96f-49f2-9d7c-c2b7d2af673b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:45.159735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:45.160533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:45.160818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:45.161124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:45.169140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:45.170740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb24cc-9dbb-4e29-8376-adcbee7efbc9 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:45.180469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 754/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:21:47.550657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60b2dd-ec3c-4be7-971a-fa5100bccd6c None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:47.551181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60b2dd-ec3c-4be7-971a-fa5100bccd6c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:47.551181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60b2dd-ec3c-4be7-971a-fa5100bccd6c None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:47.552538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 728/1482] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:47.557898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7c73-49f0-4313-9ca0-cca6cfa6b650 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:47.558299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7c73-49f0-4313-9ca0-cca6cfa6b650 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:47.558463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7c73-49f0-4313-9ca0-cca6cfa6b650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:47.585766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6e3a7c73-49f0-4313-9ca0-cca6cfa6b650 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:47.586987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6e3a7c73-49f0-4313-9ca0-cca6cfa6b650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:47.596567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 755/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:47.631785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60b2dd-ec3c-4be7-971a-fa5100bccd6c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:47.642949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60b2dd-ec3c-4be7-971a-fa5100bccd6c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:47.646657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:47.647293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:47.647582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:47.647861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:47.656644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:47.658741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3066d-7763-4426-997e-0a01443a0866 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:47.668080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 729/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:49.857565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472771d-f9c4-412c-857b-c836c9fcfe56 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:49.858209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472771d-f9c4-412c-857b-c836c9fcfe56 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:49.858209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472771d-f9c4-412c-857b-c836c9fcfe56 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:49.859505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 756/1485] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:49.865063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0165-5883-41a8-a49d-5a0fe1e6672c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:49.865523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0165-5883-41a8-a49d-5a0fe1e6672c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:49.865626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e0165-5883-41a8-a49d-5a0fe1e6672c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:49.883373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-bd2e0165-5883-41a8-a49d-5a0fe1e6672c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:49.884570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bd2e0165-5883-41a8-a49d-5a0fe1e6672c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:49.892858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 730/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:49.930724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472771d-f9c4-412c-857b-c836c9fcfe56 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:49.943171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472771d-f9c4-412c-857b-c836c9fcfe56 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:49.946620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:49.947269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:49.947559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:49.947842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:49.958666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:49.960092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a628e-0b73-4ae7-8d21-b84328fa3309 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:49.969386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 757/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:52.253408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea6884-7472-47a4-96c7-dd920475797d None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:52.253986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea6884-7472-47a4-96c7-dd920475797d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:52.253986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea6884-7472-47a4-96c7-dd920475797d None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:52.255314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 731/1488] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:52.260391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b75fd-9c51-4c1a-90b7-e7ef94b189be None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:52.260858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b75fd-9c51-4c1a-90b7-e7ef94b189be None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:52.260986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b75fd-9c51-4c1a-90b7-e7ef94b189be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:52.279254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9b3b75fd-9c51-4c1a-90b7-e7ef94b189be None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:52.280435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9b3b75fd-9c51-4c1a-90b7-e7ef94b189be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:52.289162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 758/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:21:52.330943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea6884-7472-47a4-96c7-dd920475797d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:52.341484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea6884-7472-47a4-96c7-dd920475797d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:52.345542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:52.346177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:52.346458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:52.346728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:52.355389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:52.356823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49031-ea22-4066-b0ea-2fe4c77b515f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:52.366034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 732/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:21:54.684609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1d6d1-cdcb-477a-93e9-b47afd060eac None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:54.685290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1d6d1-cdcb-477a-93e9-b47afd060eac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:54.685290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1d6d1-cdcb-477a-93e9-b47afd060eac None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:54.686580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 759/1491] 10.222.0.23 () {54 vars in 926 bytes} [Thu Jun 16 00:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:21:54.692518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7759f-f48c-4ced-ad0c-39b247aca2fc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:54.692983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7759f-f48c-4ced-ad0c-39b247aca2fc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:54.693115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7759f-f48c-4ced-ad0c-39b247aca2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:54.711138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c1e7759f-f48c-4ced-ad0c-39b247aca2fc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:21:54.712326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c1e7759f-f48c-4ced-ad0c-39b247aca2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:21:54.720273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 733/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Thu Jun 16 00:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:21:54.759502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1d6d1-cdcb-477a-93e9-b47afd060eac None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:21:54.771272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1d6d1-cdcb-477a-93e9-b47afd060eac None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:21:54.774884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:21:54.775555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:21:54.775847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:21:54.776132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:21:54.785212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:21:54.786644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74aab64-49dc-4fee-93b9-c2aa36968b0f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:21:54.796022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 760/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:22:04.705895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=33 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:22:04.791865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=33 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:23:19.894074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:19.894683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:19.894683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:19.912853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:23:19.914203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:19.922780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 734/1494] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:23:19.931904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:19.932413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:19.932520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:19.950540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:23:19.952017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:19.959964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 761/1495] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:23:19.964338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:19.973747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bffa4d-3b47-426b-8188-fb4fb8d5b842 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:19.978291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:19.978885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:19.979169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:19.979446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:19.980168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:23:19.980492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:19.981527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:19.981527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:23:19.981527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:23:19.982417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:19.994064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 735/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:23:20.002608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.011824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde1e9-09d9-4ddf-9511-e8891b4f37ba None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.017487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.018476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.018925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.019332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.020452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.023526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.023526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:23:20.023526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:23:20.025755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.068159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 762/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:23:20 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:23:20.072480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.082879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98392af4-4d5a-4234-bd1c-ba24c712e6b3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.136898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.137582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.137865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.138148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.138833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.139842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.139842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:23:20.139842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:23:20.141704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.161701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 736/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:23:20 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:23:20.165791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.175980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a4ab9-69c4-46c8-b1ea-12ce319ea46a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.180386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.181078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.181367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.181647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.182369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.183377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.183377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:23:20.183377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:23:20.184254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.197364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 763/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:23:20.201314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.211672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0b2a2-1894-408e-9c9a-6b8f176f7a64 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.216134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.216791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.217117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.217401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.218160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.219206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.219206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:23:20.219206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:23:20.220116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396b89e-7d96-440d-b1c4-a345cca08347 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.233334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 737/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:23:20.237318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.246921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34d611-21de-42c2-9065-c3c68277fe0a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.251912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.252780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.253095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.253377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] PATH_INFO: `/v3/projects/1ffa17d00d5c42c9b965ec39be03fe69/users/004e12a76e9843b6a6b64649e8b74cc7/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.269652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ffa17d00d5c42c9b965ec39be03fe69, user_id=004e12a76e9843b6a6b64649e8b74cc7, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.270696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.270696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:23:20.270696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:23:20.272850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65974d-c47c-486c-bd5d-2ccf943ed0eb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.285359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 764/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:23:20 2022] PUT /identity/v3/projects/1ffa17d00d5c42c9b965ec39be03fe69/users/004e12a76e9843b6a6b64649e8b74cc7/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:23:20.296493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b0c-523a-4a9f-8f88-373a79d1a46c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.296797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b0c-523a-4a9f-8f88-373a79d1a46c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.297101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6b0c-523a-4a9f-8f88-373a79d1a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.322897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-783e6b0c-523a-4a9f-8f88-373a79d1a46c None None] MFA Rules not processed for user `004e12a76e9843b6a6b64649e8b74cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:23:20.324178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-783e6b0c-523a-4a9f-8f88-373a79d1a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:20.386494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 738/1502] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:23:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:23:20.400179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791034-fe20-48f4-9305-a875331282a6 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.400479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791034-fe20-48f4-9305-a875331282a6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.400759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15791034-fe20-48f4-9305-a875331282a6 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.402160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 765/1503] 10.222.0.23 () {54 vars in 963 bytes} [Thu Jun 16 00:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:23:20.407541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c5f2c-f9e7-44ca-a206-e63bed723534 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.407840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c5f2c-f9e7-44ca-a206-e63bed723534 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.408123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c5f2c-f9e7-44ca-a206-e63bed723534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.426461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6b3c5f2c-f9e7-44ca-a206-e63bed723534 None None] MFA Rules not processed for user `8863fd88c1a2416c9fdb0148db03f6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:23:20.427665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6b3c5f2c-f9e7-44ca-a206-e63bed723534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:20.470741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:23:20.478930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 739/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Jun 16 00:23:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:23:20.487277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791034-fe20-48f4-9305-a875331282a6 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.498361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15791034-fe20-48f4-9305-a875331282a6 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.501978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.502613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.502898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.503175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.511298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.512705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.521394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 766/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:23:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:23:20.566972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76469-a325-4d99-8168-267631483608 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.567276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76469-a325-4d99-8168-267631483608 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.567567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a76469-a325-4d99-8168-267631483608 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.568854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 740/1506] 10.222.0.23 () {52 vars in 829 bytes} [Thu Jun 16 00:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:23:20.572468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.632648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986b121-989a-4513-a002-68d731981d10 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.637059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.637926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.638212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.638488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] PATH_INFO: `/v3/users/004e12a76e9843b6a6b64649e8b74cc7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.639645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=004e12a76e9843b6a6b64649e8b74cc7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.640688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:20.640688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:23:20.640688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:23:20.641922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.706926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004e12a76e9843b6a6b64649e8b74cc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:20.717363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004e12a76e9843b6a6b64649e8b74cc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:20.729603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004e12a76e9843b6a6b64649e8b74cc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:20.730397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004e12a76e9843b6a6b64649e8b74cc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:20.746812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:23:20.750623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004e12a76e9843b6a6b64649e8b74cc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:20.757820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 767/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:23:20 2022] DELETE /identity/v3/users/004e12a76e9843b6a6b64649e8b74cc7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:23:20.762337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.762635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.762933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.780411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:23:20.781723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:20.839244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 741/1508] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:23:20.849443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:20.850940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:20.909577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b434481-493e-4d1f-8257-f8b2c96b1a86 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:20.913191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:20.913845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:20.914130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:20.914407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:20.923933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:20.925409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db40276-8d9c-4e4d-9157-d202c6dcea56 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:20.934672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 768/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:23:21.317002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:23:21.318454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:23:21.334238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640559d-81df-4604-b2e1-d14c9145c540 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:23:21.340018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:23:21.340798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:23:21.341132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:23:21.341395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] PATH_INFO: `/v3/projects/1ffa17d00d5c42c9b965ec39be03fe69` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:23:21.342742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ffa17d00d5c42c9b965ec39be03fe69)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:23:21.344820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:23:21.344820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:23:21.344820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:23:21.346207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:23:21.423028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ffa17d00d5c42c9b965ec39be03fe69', 'request_id': 'req-f762ae1c-667c-4c58-ae4c-b252eefca2b3'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:21.436793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ffa17d00d5c42c9b965ec39be03fe69', 'request_id': 'req-f762ae1c-667c-4c58-ae4c-b252eefca2b3'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:21.443303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ffa17d00d5c42c9b965ec39be03fe69', 'request_id': 'req-f762ae1c-667c-4c58-ae4c-b252eefca2b3'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:21.458671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] The token cache is being invalidate because project 1ffa17d00d5c42c9b965ec39be03fe69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:23:21.458962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:21.459600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f762ae1c-667c-4c58-ae4c-b252eefca2b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:23:21.461292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 742/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:23:21 2022] DELETE /identity/v3/projects/1ffa17d00d5c42c9b965ec39be03fe69 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:23:30.930599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:23:31.455677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:25:00.228817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:00.229428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:00.229428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:00.230795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:00.235639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb563d-f29c-42a2-92b6-83c0c08e6182 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:00.236093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb563d-f29c-42a2-92b6-83c0c08e6182 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:00.236253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb563d-f29c-42a2-92b6-83c0c08e6182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:00.255046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-31bb563d-f29c-42a2-92b6-83c0c08e6182 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:00.256257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-31bb563d-f29c-42a2-92b6-83c0c08e6182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:00.322085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:00.499696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:00.501438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:00.579857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c169cf-3eef-4b46-928e-2d2b23c55e27 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:00.585228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:00.585891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:00.586180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:00.586461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:00.595640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:00.597821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cd4c2-effd-4165-847c-1fea82486a19 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:00.609261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 770/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:01.504052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99374c-baab-4e99-8a03-c21a435806a6 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:01.504572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99374c-baab-4e99-8a03-c21a435806a6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:01.504682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99374c-baab-4e99-8a03-c21a435806a6 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:01.506040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 744/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:01.511036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc96c5-a7a4-4382-b4e3-686ec13db2b6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:01.511497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc96c5-a7a4-4382-b4e3-686ec13db2b6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:01.511633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc96c5-a7a4-4382-b4e3-686ec13db2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:01.531608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9bdc96c5-a7a4-4382-b4e3-686ec13db2b6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:01.532816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9bdc96c5-a7a4-4382-b4e3-686ec13db2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:01.540902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 771/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:01.700648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99374c-baab-4e99-8a03-c21a435806a6 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:01.714780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99374c-baab-4e99-8a03-c21a435806a6 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:01.720981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:01.721639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:01.721933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:01.722214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:01.730617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:01.733190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa5645-4aad-4444-9918-8a0e14bd3535 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:01.742762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 745/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:01.874994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aeb2-9659-4601-9886-8aad1ac6904a None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:01.875283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aeb2-9659-4601-9886-8aad1ac6904a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:01.875565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867aeb2-9659-4601-9886-8aad1ac6904a None None] PATH_INFO: `/v3` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:01.877070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 772/1517] 10.222.0.33 () {54 vars in 925 bytes} [Thu Jun 16 00:25:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:25:01.883458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c2fa-b740-462a-8e1f-390c92e9a7df None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:01.883920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c2fa-b740-462a-8e1f-390c92e9a7df None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:01.884212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c2fa-b740-462a-8e1f-390c92e9a7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:01.904801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eaa4c2fa-b740-462a-8e1f-390c92e9a7df None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:01.906060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eaa4c2fa-b740-462a-8e1f-390c92e9a7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:01.970543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 746/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 16 00:25:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:05.815395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d255fe-eced-4b2f-85f7-2c2bf2680d66 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:05.815925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d255fe-eced-4b2f-85f7-2c2bf2680d66 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:05.816048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d255fe-eced-4b2f-85f7-2c2bf2680d66 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:05.817400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 773/1519] 10.222.0.23 () {54 vars in 960 bytes} [Thu Jun 16 00:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:05.822349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc2df-6d7c-4b57-a67d-0b0ee4a6e943 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:05.822777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc2df-6d7c-4b57-a67d-0b0ee4a6e943 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:05.822918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cc2df-6d7c-4b57-a67d-0b0ee4a6e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:05.842289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ad9cc2df-6d7c-4b57-a67d-0b0ee4a6e943 None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:05.843495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ad9cc2df-6d7c-4b57-a67d-0b0ee4a6e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:05.851946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 747/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Thu Jun 16 00:25:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:05.860318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d255fe-eced-4b2f-85f7-2c2bf2680d66 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:05.873908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d255fe-eced-4b2f-85f7-2c2bf2680d66 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:05.878935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:05.879588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:05.879860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:05.880150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:05.889036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:05.890617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8fc12-8e8f-44f8-91da-3c7d0bcc8b34 service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:05.900463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 774/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:25:13.023527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c6fa-39d3-4af2-b7f4-8ab2f5af3088 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:13.024050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c6fa-39d3-4af2-b7f4-8ab2f5af3088 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:13.024050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c6fa-39d3-4af2-b7f4-8ab2f5af3088 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:13.025489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:25:13.030234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2d2b-bec0-4ccb-984c-ba85f7b8f652 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:13.030678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2d2b-bec0-4ccb-984c-ba85f7b8f652 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:13.030785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2d2b-bec0-4ccb-984c-ba85f7b8f652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:13.048697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d26d2d2b-bec0-4ccb-984c-ba85f7b8f652 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:13.049912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d26d2d2b-bec0-4ccb-984c-ba85f7b8f652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:13.058436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:13.213756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217c6fa-39d3-4af2-b7f4-8ab2f5af3088 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:13.227799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217c6fa-39d3-4af2-b7f4-8ab2f5af3088 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:13.233614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:13.234272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:13.234557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:13.234836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:13.242763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:13.244348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c7b03-c446-4203-adbd-eab1d3f956bd service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:13.254284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 749/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:14.113752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560721e-d83f-4995-91f6-bfc006c1be60 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:14.114308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560721e-d83f-4995-91f6-bfc006c1be60 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:14.114435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560721e-d83f-4995-91f6-bfc006c1be60 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:14.115762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 00:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:14.120568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e9f39-bf72-4f6a-854c-9ed066ddb77d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:14.121019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e9f39-bf72-4f6a-854c-9ed066ddb77d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:14.121236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e9f39-bf72-4f6a-854c-9ed066ddb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:14.139611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-700e9f39-bf72-4f6a-854c-9ed066ddb77d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:14.140815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-700e9f39-bf72-4f6a-854c-9ed066ddb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:14.148812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 00:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:14.301426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560721e-d83f-4995-91f6-bfc006c1be60 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:14.316179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560721e-d83f-4995-91f6-bfc006c1be60 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:14.321515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:14.322158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:14.322448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:14.322731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:14.331323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:14.332927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4985c9c-73fb-4cc0-861d-68f13f773ed1 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:14.342507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 777/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:18.706592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:18.707129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:18.707241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:18.708541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:18.713870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58828a7-b261-42b2-a79d-c295211ad986 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:18.714305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58828a7-b261-42b2-a79d-c295211ad986 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:18.714456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58828a7-b261-42b2-a79d-c295211ad986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:18.733829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c58828a7-b261-42b2-a79d-c295211ad986 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:18.735022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c58828a7-b261-42b2-a79d-c295211ad986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:18.742742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:18.783048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:18.784800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:18.840891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf623a-53a7-4b5d-919c-fa9534b8b8ce None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:18.845208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:18.845860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:18.846148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:18.846429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:18.855434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:18.857005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73324848-2883-49ab-9077-58cd594d712e service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:18.862615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:25:18.867699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 752/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:20.954791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c116-39fe-4072-b167-e026c3220da2 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:20.955329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c116-39fe-4072-b167-e026c3220da2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:20.955446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c116-39fe-4072-b167-e026c3220da2 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:20.956822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:20.962108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd82e-ea91-45c5-a6cf-9273449ae113 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:20.962529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd82e-ea91-45c5-a6cf-9273449ae113 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:20.962690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd82e-ea91-45c5-a6cf-9273449ae113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:20.981277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-780cd82e-ea91-45c5-a6cf-9273449ae113 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:20.983080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-780cd82e-ea91-45c5-a6cf-9273449ae113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:20.991197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:21.027217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09c116-39fe-4072-b167-e026c3220da2 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:21.033052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=56 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:25:21.039275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09c116-39fe-4072-b167-e026c3220da2 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:21.042745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:21.043382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:21.043668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:21.043947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:21.053312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:21.054735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f9f1e-0dec-4090-b706-3af9875a7567 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:21.063992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 780/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:24.678022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:24.678588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:24.678588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:24.679963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:24.685055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c58-64f8-4631-a8ff-52b558784d89 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:24.685492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c58-64f8-4631-a8ff-52b558784d89 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:24.685668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b8c58-64f8-4631-a8ff-52b558784d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:24.703154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c20b8c58-64f8-4631-a8ff-52b558784d89 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:24.704466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c20b8c58-64f8-4631-a8ff-52b558784d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:24.712212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:24.983457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:24.985188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:25.042187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e696-d391-41d3-9a7d-fe2ae3f77394 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:25.045894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:25.046535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:25.046818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:25.047098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:25.055798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:25.057253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ea12e-b33a-48fa-924d-4538e964e8ec service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:25.066606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 755/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:27.213754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:27.214303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:27.214425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:27.215731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 00:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:25:27.220907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a4787-edbd-42a3-8cb9-dc23680cd70e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:27.221338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a4787-edbd-42a3-8cb9-dc23680cd70e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:27.221488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a4787-edbd-42a3-8cb9-dc23680cd70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:27.240798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d50a4787-edbd-42a3-8cb9-dc23680cd70e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:25:27.242087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d50a4787-edbd-42a3-8cb9-dc23680cd70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:27.250619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 00:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:25:27.512505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:25:27.514737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:25:27.527930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637a393-c3fc-4a82-b922-5c7349a7d046 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:25:27.532088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:25:27.532731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:25:27.533074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:25:27.533361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:25:27.542229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:25:27.543656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8e86d-631f-49c2-bff9-493caa94c703 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:25:27.553145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 783/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:25:37.234235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=32 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:25:37.548962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=14 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:57.304460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.305642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.306168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.334566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:57.336141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:57.352001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 757/1540] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:57.361758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.362777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.363286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.392018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:57.393630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:57.411260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 784/1541] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:57.416919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.434862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3dd6b-f50d-4c17-be7c-6c74a15c7f8b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.442951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.443885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.444390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.444909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.445876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:57.446414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.453495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.469984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 758/1542] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:57.478834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.489202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56b7e-47dd-496c-9e6e-ea005c546fc1 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.500769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.501703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.502152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.502596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.503935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.506085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.562333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 785/1543] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:57.566897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.584486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae46aa-b037-45e5-814b-b25089f50bc0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.670782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.671898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.672408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.672924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.673872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.676014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.705989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 759/1544] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:57.710201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.730314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db581544-d019-4436-b004-c869295e4a9c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.740058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.740763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.741077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.741413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.742225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.745421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.763176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 786/1545] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:57.767266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.784962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084521e-20cb-48e8-bd11-f98ede380b2c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.798680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.799821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.800339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.800865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.801890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.803793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b86601-c8ae-451d-a164-1abda2bc4058 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.819704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 760/1546] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:57.825121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:57.841314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decfbb3-a052-466f-a5aa-8577f87dc56c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:57.845073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eab3a-a1c4-4a1f-943b-d448713d4fc0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.845631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eab3a-a1c4-4a1f-943b-d448713d4fc0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.846136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293eab3a-a1c4-4a1f-943b-d448713d4fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.846987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:57.847897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.848182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.848463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] PATH_INFO: `/v3/projects/83a157616f11430fa989f95eeeeea643/users/825b769623d94188be82535a7641628d/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.876778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-293eab3a-a1c4-4a1f-943b-d448713d4fc0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:57.877244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a157616f11430fa989f95eeeeea643, user_id=825b769623d94188be82535a7641628d, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:57.878610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-293eab3a-a1c4-4a1f-943b-d448713d4fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:57.879950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0761566-8dcb-4ade-97e2-ebdccbfd0b86 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:57.901472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 761/1547] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 00:26:57.902997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea753-78b7-46c0-b383-d0499d05db71 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.903390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea753-78b7-46c0-b383-d0499d05db71 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.903784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea753-78b7-46c0-b383-d0499d05db71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.909555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 787/1548] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:57 2022] PUT /identity/v3/projects/83a157616f11430fa989f95eeeeea643/users/825b769623d94188be82535a7641628d/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:57.913354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2e8a-cf85-47d3-9572-c0f2eeb3a34a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:57.913823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2e8a-cf85-47d3-9572-c0f2eeb3a34a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:57.914272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad2e8a-cf85-47d3-9572-c0f2eeb3a34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:57.930705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f56ea753-78b7-46c0-b383-d0499d05db71 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:57.933410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f56ea753-78b7-46c0-b383-d0499d05db71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:57.945324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c1ad2e8a-cf85-47d3-9572-c0f2eeb3a34a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:57.946872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c1ad2e8a-cf85-47d3-9572-c0f2eeb3a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.060024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 762/1549] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.061330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8674e-f2b8-44da-b5e1-bf94d896fba4 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.061627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8674e-f2b8-44da-b5e1-bf94d896fba4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.061910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8674e-f2b8-44da-b5e1-bf94d896fba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.073440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 788/1550] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.076335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.076335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.076335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.093175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b7e8674e-f2b8-44da-b5e1-bf94d896fba4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.094489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b7e8674e-f2b8-44da-b5e1-bf94d896fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.105869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 763/1551] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.107163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e56-5316-4e60-b461-1c57d0d3e7f4 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.107740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e56-5316-4e60-b461-1c57d0d3e7f4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.108047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d82e56-5316-4e60-b461-1c57d0d3e7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.109146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] MFA Rules not processed for user `825b769623d94188be82535a7641628d`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.110712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.128243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-52d82e56-5316-4e60-b461-1c57d0d3e7f4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.129589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-52d82e56-5316-4e60-b461-1c57d0d3e7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.144845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 764/1552] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.144845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03ea5-f877-4edd-a585-0d4f8978e9d2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.144845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03ea5-f877-4edd-a585-0d4f8978e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.144845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03ea5-f877-4edd-a585-0d4f8978e9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.170885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-66a03ea5-f877-4edd-a585-0d4f8978e9d2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.172544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-66a03ea5-f877-4edd-a585-0d4f8978e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.182370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 765/1553] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.183741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f275b-f3e8-4740-9dca-77efe250a8e3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.184072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f275b-f3e8-4740-9dca-77efe250a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.184377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f275b-f3e8-4740-9dca-77efe250a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.205155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-5b5f275b-f3e8-4740-9dca-77efe250a8e3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.205915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 789/1554] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.206795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.209740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5b5f275b-f3e8-4740-9dca-77efe250a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.217764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec80491c-c939-44c9-ae4d-563b8f407c03 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.219311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 766/1555] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.220574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579b23-50e9-4772-aa60-68504481e330 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.220859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579b23-50e9-4772-aa60-68504481e330 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.221189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36579b23-50e9-4772-aa60-68504481e330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.227942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.229283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.229576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.229859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.230598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.230912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.231947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:26:58.231947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:26:58.231947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:26:58.232855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e89c36-4ab4-4e92-a303-beaba7327b90 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.243236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-36579b23-50e9-4772-aa60-68504481e330 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.244628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-36579b23-50e9-4772-aa60-68504481e330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.245066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 790/1556] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.246354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4aba35-3ece-4765-8cdb-7b85d15bc755 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.246662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4aba35-3ece-4765-8cdb-7b85d15bc755 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.247049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4aba35-3ece-4765-8cdb-7b85d15bc755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.254667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 767/1557] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.255199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.255493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.255772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.266394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-2f4aba35-3ece-4765-8cdb-7b85d15bc755 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.267658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2f4aba35-3ece-4765-8cdb-7b85d15bc755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.277320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.277645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 791/1558] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.285202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.285317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.285317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.285317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.291711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 768/1559] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.292537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.293042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:58.299444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.301036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.302449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c9304-942a-4d46-bd32-bbdac13e9021 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.307110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.307768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.308057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.308375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.309288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 792/1560] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.309400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.309477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.309885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.311128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d304364-2378-4280-b161-be72f60bd798 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.319723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6f402-4b81-45a3-925e-85b500c68b77 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.320584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:58.325050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 769/1561] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.325759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.326439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.326590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.326741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.326888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.326999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.327172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.328030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.330083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.352798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.353941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.359149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 793/1562] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.361871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.362575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 770/1563] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.362690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.372976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a6157-99f9-4f3b-a914-0dc0d4eb01e4 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.426140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb839a0-2749-421c-89d0-cbd01f4afbfc admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.429372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.430074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.430362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.430642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.430889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.431945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.432132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.432320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.432449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.432607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.433339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.434083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4459428-2ce5-411c-956f-7124a84edaa8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.435259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.446819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 771/1564] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.448685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2ea5-e73f-4844-b18d-274145f61f59 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.449014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2ea5-e73f-4844-b18d-274145f61f59 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.449305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d2ea5-e73f-4844-b18d-274145f61f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.466168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 794/1565] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.467031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.470303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b33d2ea5-e73f-4844-b18d-274145f61f59 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.471539 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b33d2ea5-e73f-4844-b18d-274145f61f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.531638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87a779-516a-4eec-8801-dafc75d9d453 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.533930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 772/1566] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.535217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.535515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.535799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.536215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.536867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.537230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.537452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.538440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.538756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.540385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.557151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 795/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.557151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.558504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:58.560419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:58.572113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255fea8-c625-45bc-ab9e-8a41d53a9622 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.577168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 773/1568] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:58.577168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.580945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.581727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.582014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.582317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.583093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.583719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f019c-1725-4895-8a22-ea2af22fdb83 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.585031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.588278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.588950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.589244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.589525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.590215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.592120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.613067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 796/1569] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.616433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 774/1570] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/users => generated 299 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.616541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.616637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.626652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9ce96-850e-4267-8276-7c5f17d36d97 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.656208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:58.680350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5831c2-65ac-4bc7-a4ee-4ab6e2d2eed0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.685542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.685744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.686212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.686409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.686514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.686699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.686799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.686980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.687495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.687673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.687812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.688700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:26:58.688700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:26:58.688700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:26:58.689436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.689916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.698950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:58.701363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 797/1571] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.702158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.712761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8191d0-f923-4870-b441-b151de46469e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.719266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.719997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.720290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.720598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.721328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.722357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:26:58.722357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:26:58.722357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:26:58.723569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.727812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 775/1572] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.727812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.743095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 798/1573] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.744414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.753740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a907a95-a2bc-41c6-8f11-b386be17866f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.795253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fb0cf-c0a8-48b6-bb6a-3214a0d252f1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.799871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.800534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.800820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.801133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.801855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.802164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.803753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.811340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.812017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.812310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.812592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.813326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.815274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.816074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 776/1574] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.821916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.831776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfea21-f6da-4fac-9dae-621c5bdaf74e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.836783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.837508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.837969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.838257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.838986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:26:58.839302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.841024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.845183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 799/1575] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.845183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.864681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 777/1576] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:26:58.864681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.868526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5a7f4-d3f9-47ba-aa54-b2059e8ed70d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.904087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e075f9c-df85-4b9e-a647-c22714d36bbe admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.908853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.909557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.909849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.910124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.910842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.912894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.926750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.927405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.927765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.928005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.928797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.930430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.933674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 800/1577] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.934546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.942141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 778/1578] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:58.943036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.947554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccac1f-09a3-4b84-8041-05abb7ee73e7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.952462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.953116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f67e13-ec3b-46f1-8aeb-4eeca79e6846 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:58.953239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.953513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.953868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.954507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.956450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.957847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:58.958494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:58.958778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:58.959064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:58.959754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:58.961682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:58.982734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 801/1579] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.983440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 779/1580] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:58.983578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.984246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:58.993415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:58.994288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d417c7-2b3f-40f1-ac22-03aa32afee39 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.041340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:59.053924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a34a9c-a127-47a7-a46f-4273b9f677b2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.054887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.055558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.055847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.056131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.056940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.058873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.058985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.059643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.059938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.060223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.061112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.062765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.076190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 802/1581] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.076190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.083976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf110f-0e38-4879-afc8-0376737659b3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.089052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.089707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.089997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.090277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.090698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 780/1582] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:58 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.090963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.091719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.092891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.112799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 803/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/users => generated 297 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.114071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.124409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452e9c7-f4e3-4f03-acb0-8a1a95b3dfd8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.163243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213aa01-7561-4da2-a1a6-764d8bc2dc94 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.168020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.168689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.169012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.169304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.170001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.171908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.184239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.184939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.185237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.185514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.186242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.188078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.201219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 781/1584] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.201219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.201503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 804/1585] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.201503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.211248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac450d-796f-4dee-a8be-0dfabcc2f5f8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.265467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd555f-4fef-4a38-bbc7-b6b5d4333d2e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.267280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.267959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.268245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.268539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.269306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.270332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.270928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.271463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.271755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.272073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] PATH_INFO: `/v3/projects/9af43eefaed34994bddbbbc9246de574/users/e59f8799a46b43b7b180fd599f96dd6a/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.281194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 805/1586] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.282092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.287751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9af43eefaed34994bddbbbc9246de574, user_id=e59f8799a46b43b7b180fd599f96dd6a, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.288788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:26:59.288788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:26:59.288788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:26:59.290841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.291613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2337-aabd-4727-b5cf-5dd08ace3748 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.296686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.297494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.297913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.298225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.299126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.301038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.305026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 782/1587] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/9af43eefaed34994bddbbbc9246de574/users/e59f8799a46b43b7b180fd599f96dd6a/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:59.305826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.316473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:59.326320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 806/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.326320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.334457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaea45-0ed0-47cb-bf70-d5e680c29532 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.355039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:59.373210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a898a-eba2-4672-9cfe-a33891cddf3a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.381234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.381909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.382194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.382470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.383205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.385134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.390751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.391468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.391760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.392043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.392741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.394731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.395475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 783/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.396275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.407060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef1345-8ee9-4266-95f8-066a617ee231 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.412534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.413249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.413560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.413819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.414599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.416497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 807/1590] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.416918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.418025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.428152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 784/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.429043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f3688-ded5-4251-aa6b-332f9b53e57b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.429156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.434936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.435644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.435928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.436212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.436956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.438871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.440901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b74d-8d30-4fb8-8d2c-96d5ee661686 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.445559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.446077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.446369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.446652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.447883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.449911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.459197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 808/1592] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:26:59.460192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.462505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 785/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.463856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.471119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d591d95-d936-4fd5-9751-5986841e2a4a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.474793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc31f2-227b-4280-a2f4-73a40babff12 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.475818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.476582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.476849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.477158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.477892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.480339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.481023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.481320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.481602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.482502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.483775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.484066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.494236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 786/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.495175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.498509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 809/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.499335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.505128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadfbef-fad2-4c3a-b233-eb36c46eea77 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.508852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66cf5-e995-4787-a092-8d25d397d0b8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.510552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.511251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.511563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.511843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.512566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.514187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.514664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.515531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.515817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.516344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] PATH_INFO: `/v3/projects/8256b9c4822941639662d00143e57bf4/users/266a74e9540049eea14ea6a46052cd0e/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.524844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 787/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.525913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.534715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8256b9c4822941639662d00143e57bf4, user_id=266a74e9540049eea14ea6a46052cd0e, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.535829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896e1b5-84ec-425b-87d7-be57ac6ac657 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.537415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.540376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.541929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.542903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.542903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.543734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.545669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.551667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 810/1597] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/8256b9c4822941639662d00143e57bf4/users/266a74e9540049eea14ea6a46052cd0e/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:59.552706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.564924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 788/1598] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.565947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.578053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c1-d818-496f-b78c-0667e0696b88 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.617887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650554-87a6-4626-bc92-e8daddb8cbfb admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.622742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.623460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.623791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.624072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.624798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.626422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.633538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.634223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.635048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.635333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.636070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.637680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.641464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 811/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.641464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.648096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 789/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.648872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.654102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06433534-2ce0-4342-95d4-cbb2b2bfc224 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.658710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c879-18d5-43d6-96c3-2d979f182079 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.659220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.660179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.660484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.660963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] PATH_INFO: `/v3/projects/3bf50ba5c25c4892ae4ecea7f5346e65/users/193aeba5796440c7bda593fb719ec0d7/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.663218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.664089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.664374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.664654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] PATH_INFO: `/v3/projects/9af43eefaed34994bddbbbc9246de574/users/e59f8799a46b43b7b180fd599f96dd6a/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.666698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9af43eefaed34994bddbbbc9246de574, user_id=e59f8799a46b43b7b180fd599f96dd6a, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.669356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.677072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf50ba5c25c4892ae4ecea7f5346e65, user_id=193aeba5796440c7bda593fb719ec0d7, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.679593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 790/1601] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/9af43eefaed34994bddbbbc9246de574/users/e59f8799a46b43b7b180fd599f96dd6a/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:59.680358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.681061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.696125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 812/1602] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/3bf50ba5c25c4892ae4ecea7f5346e65/users/193aeba5796440c7bda593fb719ec0d7/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:59.697861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.710125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:59.749507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660e709-246c-49f6-abbf-95025757966f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.758434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:26:59.768554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90d8b8-173b-49d1-af52-a771d4528f64 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.773454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.774120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.774407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.774680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.775491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.777108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b488c42-e7f7-4397-bf83-8890379377d2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.786763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 813/1603] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.788007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.788298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.788580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.809500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.810438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.810753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.811055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] PATH_INFO: `/v3/projects/7e7ced4b21544bd689b6b0cae01566b6/users/01c74d394beb4d7286563fc4a9687d50/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.817738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] MFA Rules not processed for user `266a74e9540049eea14ea6a46052cd0e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:26:59.819023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:26:59.826525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7ced4b21544bd689b6b0cae01566b6, user_id=01c74d394beb4d7286563fc4a9687d50, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.829283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.843672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 791/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/7e7ced4b21544bd689b6b0cae01566b6/users/01c74d394beb4d7286563fc4a9687d50/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:26:59.845810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.885699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 814/1605] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:26:59.886558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.899831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09357-9e4d-4e0c-a1b1-54c2a3a1dd21 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.919203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c860c-16a3-42fd-b254-99e746d14056 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.926343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.927208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.927635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.927953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.928773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.930513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.940644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 792/1606] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.941593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:26:59.951279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489008f5-b8ec-40ac-a74d-845c03f4fc98 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:26:59.955878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.956556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.956851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.957280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.957894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.959531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf910b0f-35ee-4b30-b7ea-4d698062329d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.970092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 793/1607] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:26:59.971305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.971412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:26:59.971606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.971921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.972469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:26:59.972829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:26:59.973456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] PATH_INFO: `/v3/projects/0e5f17f8ba8e4113a27a004ebb29917a/users/b5f4125ecaec41e5ba5c4eaf00c43e07/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:26:59.990324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5f17f8ba8e4113a27a004ebb29917a, user_id=b5f4125ecaec41e5ba5c4eaf00c43e07, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:26:59.993007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6d1f1-9ef0-43da-9ca2-b5e75ac5d8be admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:26:59.999549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] MFA Rules not processed for user `e59f8799a46b43b7b180fd599f96dd6a`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.000928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.007645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 815/1608] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:26:59 2022] PUT /identity/v3/projects/0e5f17f8ba8e4113a27a004ebb29917a/users/b5f4125ecaec41e5ba5c4eaf00c43e07/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:00.010493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5606e1-4e65-47cb-ae76-a37a21654d85 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.010894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5606e1-4e65-47cb-ae76-a37a21654d85 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.011205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5606e1-4e65-47cb-ae76-a37a21654d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.028811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7c5606e1-4e65-47cb-ae76-a37a21654d85 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.030034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7c5606e1-4e65-47cb-ae76-a37a21654d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.038960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.072813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 794/1609] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:26:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.073915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.082853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.090845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 816/1610] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.092153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad48-cd08-40b4-b130-c403cd0e47eb None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.092451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad48-cd08-40b4-b130-c403cd0e47eb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.092735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad48-cd08-40b4-b130-c403cd0e47eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.110832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d14fad48-cd08-40b4-b130-c403cd0e47eb None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.112110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d14fad48-cd08-40b4-b130-c403cd0e47eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.119977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 817/1611] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.121183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fd2ac-87c5-424e-bd41-82869f4509a3 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.121481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fd2ac-87c5-424e-bd41-82869f4509a3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.121770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fd2ac-87c5-424e-bd41-82869f4509a3 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.123787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 818/1612] 10.222.0.23 () {54 vars in 961 bytes} [Thu Jun 16 00:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:00.124177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fd2ac-87c5-424e-bd41-82869f4509a3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.137046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fd2ac-87c5-424e-bd41-82869f4509a3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.141657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.142617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.142997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.143353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] PATH_INFO: `/v3/projects/9eeb911e348948edaccb0f977a645c39/users/cd12200a054240e389eca80f2c16f59c/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.144610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb11c-bf2b-4569-b3c9-84bc7573f62f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.150838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.151732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.152019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.152298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] PATH_INFO: `/v3/projects/e6364c3ea3274ff690cd68f68f215784/users/097d0b953daf4d75b2cbf94f9324915c/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.158863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eeb911e348948edaccb0f977a645c39, user_id=cd12200a054240e389eca80f2c16f59c, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.161556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.167708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6364c3ea3274ff690cd68f68f215784, user_id=097d0b953daf4d75b2cbf94f9324915c, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.170650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa88e55-4739-4db6-a685-cc2185b34cc4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.175294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 819/1613] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:00 2022] PUT /identity/v3/projects/9eeb911e348948edaccb0f977a645c39/users/cd12200a054240e389eca80f2c16f59c/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:00.176280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.192134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 795/1614] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:00 2022] PUT /identity/v3/projects/e6364c3ea3274ff690cd68f68f215784/users/097d0b953daf4d75b2cbf94f9324915c/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:00.193441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.193734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.194017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.210936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.212256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.247846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af3373-5495-4078-bd34-0d17f2c65bc1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.273396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 796/1615] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.274248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.307396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.308285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.308582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.308862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] PATH_INFO: `/v3/projects/cd8526627add42a597eff6fce7400885/users/f1d7bb8f9d364c5198313fedd982478a/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.323557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8526627add42a597eff6fce7400885, user_id=f1d7bb8f9d364c5198313fedd982478a, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.326370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.336434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec064e5-0f37-411c-aab2-72fb9de2e7a3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.339838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 820/1616] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:00 2022] PUT /identity/v3/projects/cd8526627add42a597eff6fce7400885/users/f1d7bb8f9d364c5198313fedd982478a/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:00.344349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.364091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.381879 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.388853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.389938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.390451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.390746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.405528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f18f7-9c0e-4466-b98a-82ca45070bd3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.409498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.409807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.410091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.410371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.452627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.454124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6e07f-a4c0-4d56-a616-efcbabcce024 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.464221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 797/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.465584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf761b-1a89-44d7-ab6e-56f30ea0e9f9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.465957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf761b-1a89-44d7-ab6e-56f30ea0e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.466203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf761b-1a89-44d7-ab6e-56f30ea0e9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.473255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.474736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.483386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-35cf761b-1a89-44d7-ab6e-56f30ea0e9f9 None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.484585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-35cf761b-1a89-44d7-ab6e-56f30ea0e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.485536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 821/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.489783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.491119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.502796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c06f-2451-4e40-8ecd-eace77842117 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.507150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.507825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.508121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.508399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.520838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.523927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57eb7f-cf62-4d37-8062-3b9edfa03f2f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.536985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 822/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.538359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905450-0930-48e5-a533-d154288e473f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.538658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905450-0930-48e5-a533-d154288e473f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.539107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905450-0930-48e5-a533-d154288e473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.580222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-fe905450-0930-48e5-a533-d154288e473f None None] MFA Rules not processed for user `cd12200a054240e389eca80f2c16f59c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.582981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fe905450-0930-48e5-a533-d154288e473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.628408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 798/1620] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.639301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.639573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.640567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.687612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] MFA Rules not processed for user `097d0b953daf4d75b2cbf94f9324915c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.689277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.710852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 823/1621] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.711937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe905450-0930-48e5-a533-d154288e473f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.738527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe905450-0930-48e5-a533-d154288e473f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.750229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.753652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.754181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.755318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.777594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.780193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844e604-b096-4e7b-8ad0-27f6e1918b0f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.797738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 824/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.799675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.801618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.802324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.831788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:00.833201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 799/1623] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.833858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:00.834144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.846211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 825/1624] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:00.847083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:00.858009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cce2b0-9261-4117-ac8f-48a66de98e93 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.858164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cea1b-84f8-4346-89fa-816eaa3c2d9c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:00.863021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.864022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.864417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.864816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] PATH_INFO: `/v3/users/825b769623d94188be82535a7641628d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.865892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:00.866447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] RBAC: Authorizing `identity:delete_user(user_id=825b769623d94188be82535a7641628d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.867445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.867803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.868087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.868587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.891409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.892989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:00.976637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825b769623d94188be82535a7641628d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:00.980306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:00.981975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7d903-33bc-414d-80ca-4338b99895d3 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:00.988158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825b769623d94188be82535a7641628d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:00.995656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 800/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:00.997453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849dd2e-c3e0-491d-8a3f-21b55db258dc None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:00.997706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825b769623d94188be82535a7641628d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:00.997839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849dd2e-c3e0-491d-8a3f-21b55db258dc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:00.998076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849dd2e-c3e0-491d-8a3f-21b55db258dc None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:00.999002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825b769623d94188be82535a7641628d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:00.999765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 801/1626] 10.222.0.23 () {54 vars in 961 bytes} [Thu Jun 16 00:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:01.000778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849dd2e-c3e0-491d-8a3f-21b55db258dc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:01.014174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825b769623d94188be82535a7641628d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:01.017957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849dd2e-c3e0-491d-8a3f-21b55db258dc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:01.022731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:01.023678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.023907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 826/1627] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:00 2022] DELETE /identity/v3/users/825b769623d94188be82535a7641628d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:01.024032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.024504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.026663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:01.027928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.029148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.045997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:01.048503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81c62d-804f-4e5d-a7aa-6f4c2fbec905 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:01.060601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 802/1628] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:01.061899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87cac0-8d6e-4468-9541-a2b4832b4ab2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.062320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87cac0-8d6e-4468-9541-a2b4832b4ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.062877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87cac0-8d6e-4468-9541-a2b4832b4ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.084221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8a87cac0-8d6e-4468-9541-a2b4832b4ab2 None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:01.085721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a87cac0-8d6e-4468-9541-a2b4832b4ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.101074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc50bd3-5c25-4ecb-8355-9052a42e690b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:01.106009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:01.106754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.107319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.107970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.110258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.161611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 803/1629] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Jun 16 00:27:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:01.163399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.163748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.164035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.191517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:01.192842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.195418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:01.199136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:01.216681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 804/1630] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:01.218196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 827/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:01.219101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:01.230225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:01.231921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.247483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881a9c-7efe-46e7-9ad2-500cae4ff033 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:01.262236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d877e-3d97-4843-9ebb-c95c285cdc33 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:01.268687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:01.268860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:01.269831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.270335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.270822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.272677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.273263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.273555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.273887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.297060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:01.299635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f925974-bfb5-4a9c-9a6b-13115c3ddc98 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:01.333355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 828/1632] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:01.380591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a2d8a-afd0-4bb3-9eb3-10b719b578a6 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.380912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a2d8a-afd0-4bb3-9eb3-10b719b578a6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.381190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a2d8a-afd0-4bb3-9eb3-10b719b578a6 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.382951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 829/1633] 10.222.0.23 () {54 vars in 912 bytes} [Thu Jun 16 00:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:27:01.390367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e87807-ddcf-4ba3-a1f5-737a1a02c0cb None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.390700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e87807-ddcf-4ba3-a1f5-737a1a02c0cb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.391660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e87807-ddcf-4ba3-a1f5-737a1a02c0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.421471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:01.425355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:01.425801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-03e87807-ddcf-4ba3-a1f5-737a1a02c0cb None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:01.428012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-03e87807-ddcf-4ba3-a1f5-737a1a02c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.453713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 805/1634] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:01.464386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 830/1635] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:01.481352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:01.483893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.535354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:01.574624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce8ff2-7a34-4b85-b52a-7339e920e99b service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:01.579927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:01.580810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.581962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.582256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.601836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:01.603392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cdea4-0a80-42b3-b4e6-b7d80359d357 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:01.631773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 806/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:01.923904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.927814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.928537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.931599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 831/1637] 10.222.0.23 () {54 vars in 912 bytes} [Thu Jun 16 00:27:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:01.938042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:01.938340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:01.939427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:01.970147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:01.971391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:01.989221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 807/1638] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:02.006501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:02.011933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:02.058809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78855e6d-d5ab-4af3-91ee-c066ee9e5e16 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:02.075182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:02.077982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:02.080648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:02.082457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:02.132515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:02.139322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4c607-e7a8-4d03-9169-4408ccab8682 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:02.160334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 832/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:02.467686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:02.470150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:02.517931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb66e43-6009-456f-a041-fa7e69621adb None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:02.524121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:02.525675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:02.526331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:02.526833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] PATH_INFO: `/v3/projects/83a157616f11430fa989f95eeeeea643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:02.528313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a157616f11430fa989f95eeeeea643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:02.530786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:02.606404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a157616f11430fa989f95eeeeea643', 'request_id': 'req-d9d4b496-6603-4fbe-ab78-fa6f13717dae'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:02.621142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:02.621826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:02.622466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:02.641206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a157616f11430fa989f95eeeeea643', 'request_id': 'req-d9d4b496-6603-4fbe-ab78-fa6f13717dae'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:02.656093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a157616f11430fa989f95eeeeea643', 'request_id': 'req-d9d4b496-6603-4fbe-ab78-fa6f13717dae'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:02.668865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:02.685302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] The token cache is being invalidate because project 83a157616f11430fa989f95eeeeea643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:02.685586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:02.686604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:02.690525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] MFA Rules not processed for user `193aeba5796440c7bda593fb719ec0d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:02.691632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 808/1640] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:02 2022] DELETE /identity/v3/projects/83a157616f11430fa989f95eeeeea643 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:02.693281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:02.693906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:02.697712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:02.871246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 833/1641] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:02.876218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:02.878967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:02.896800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b496-6603-4fbe-ab78-fa6f13717dae admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:02.915125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:02.919640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:02.921361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:02.921782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] PATH_INFO: `/v3/users/e59f8799a46b43b7b180fd599f96dd6a` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:02.924450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a28c8-77a3-45b8-b8a8-51c4811d1523 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:02.925615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59f8799a46b43b7b180fd599f96dd6a)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:02.926692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: WARNING py.warnings [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:27:02.926692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: warnings.warn(msg) Jun 16 00:27:02.926692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: Jun 16 00:27:02.928323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:02.945569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:02.946046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:02.946342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:02.946658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:02.950739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:02.953074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.018626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 834/1642] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:02 2022] POST /identity/v3/projects => generated 401 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:03.021849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.091179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59f8799a46b43b7b180fd599f96dd6a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:03.093583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:03.107626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59f8799a46b43b7b180fd599f96dd6a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:03.184425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59f8799a46b43b7b180fd599f96dd6a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:03.185429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59f8799a46b43b7b180fd599f96dd6a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:03.187442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59f8799a46b43b7b180fd599f96dd6a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:03.212445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c9fc9-27fa-4d37-b636-91d6fc120876 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.222401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 809/1643] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:02 2022] DELETE /identity/v3/users/e59f8799a46b43b7b180fd599f96dd6a => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:03.224547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.225732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.226064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.226652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.227078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.229884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.259091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 835/1644] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:03 2022] POST /identity/v3/users => generated 307 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:03.263554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.265363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:03.382385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2259-9dab-4bd4-9592-6f8dc5a814c6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.392152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.393113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.393535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.393911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.394808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.398963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.414609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 810/1645] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:03.419720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.441209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88551508-0b39-40f0-be05-2ca813f381e7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.451175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.456617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.457415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.459523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] PATH_INFO: `/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d/users/cb1342ce5ad343a190bbfc9986adbf0b/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.506162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8509e01e3fb497c8a705c003e0c7c4d, user_id=cb1342ce5ad343a190bbfc9986adbf0b, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.509616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.541267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 836/1646] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:03 2022] PUT /identity/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d/users/cb1342ce5ad343a190bbfc9986adbf0b/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:03.546057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.563813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d71-17c8-401d-8072-bb2e990c1086 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.710513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.711445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.711774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.712113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.713759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.715489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.738108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 811/1647] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:03.742169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.758612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b4f58-a22b-444d-b640-f75ee2abdc9d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.766081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.767188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.767729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.768736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] PATH_INFO: `/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d/users/cb1342ce5ad343a190bbfc9986adbf0b/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.772089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8509e01e3fb497c8a705c003e0c7c4d, user_id=cb1342ce5ad343a190bbfc9986adbf0b, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.778525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9811fc1-73ce-4076-90e5-e2387edaa4d4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:03.794000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 837/1648] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:03 2022] PUT /identity/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d/users/cb1342ce5ad343a190bbfc9986adbf0b/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:03.798137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:03.800612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:03.831555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0347-41ee-451b-984b-285cc88ac2e1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:03.933456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:03.946213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:03.947628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:03.948349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:03.949074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] PATH_INFO: `/v3/projects/9af43eefaed34994bddbbbc9246de574` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:03.950922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9af43eefaed34994bddbbbc9246de574)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:03.954220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.041534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9af43eefaed34994bddbbbc9246de574', 'request_id': 'req-24f34216-40c9-4d36-90f0-f7e9140874ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:04.052217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9af43eefaed34994bddbbbc9246de574', 'request_id': 'req-24f34216-40c9-4d36-90f0-f7e9140874ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:04.069112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9af43eefaed34994bddbbbc9246de574', 'request_id': 'req-24f34216-40c9-4d36-90f0-f7e9140874ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:04.097312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] The token cache is being invalidate because project 9af43eefaed34994bddbbbc9246de574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:04.097661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:04.099112 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24f34216-40c9-4d36-90f0-f7e9140874ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:04.103252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 812/1649] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:03 2022] DELETE /identity/v3/projects/9af43eefaed34994bddbbbc9246de574 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:04.108073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.108365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.108916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.159387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:04.160650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:04.238430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:04.287278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 838/1650] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:04.295548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.296177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.296664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.392423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:04.393666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:04.424029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 813/1651] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:04.428198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.443237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e044cc-721e-4b78-9103-93e8975206b3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.462208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.463773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.464256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.464561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.465365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:04.465645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:04.467255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.508279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 839/1652] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:04.512743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.527349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107bf4c-e10d-49a0-b5d4-5cad0631e836 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.538132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.538837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.539110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.539658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.540372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:04.542961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.606459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 814/1653] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:04 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:04.610867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.628911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574d17-78bc-4d3f-b6e3-a13921d8e1d5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.744043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.745263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.745561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.745847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.746930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:04.749069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.779943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 840/1654] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:04 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:04.781235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.796520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aebb10-1a66-4e05-89e9-f9448cab5baa admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.811416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.812509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.812964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.813285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.814413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:04.822683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.844531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 815/1655] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:04.848262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.866944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf21ff-bb56-4d0e-94c2-f282e8fa55c2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.875673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.876922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.877692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.878795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.879870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:04.882717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496704-fde1-4bd3-a33e-e2f762c62c0d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:04.898538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 841/1656] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:04.902571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:04.917119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71c51e-f63b-4c17-a9e0-6d2780112178 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:04.932334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:04.933602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:04.934123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:04.934568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] PATH_INFO: `/v3/projects/0c32911b5f104dcba4cad89bfc78769d/users/10e68a06b740494998459da91fe1f8e7/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:04.997205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c32911b5f104dcba4cad89bfc78769d, user_id=10e68a06b740494998459da91fe1f8e7, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:05.000373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:05.028936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 816/1657] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:04 2022] PUT /identity/v3/projects/0c32911b5f104dcba4cad89bfc78769d/users/10e68a06b740494998459da91fe1f8e7/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:05.037529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:05.038037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:05.039035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:05.090000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] MFA Rules not processed for user `10e68a06b740494998459da91fe1f8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:05.091241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:05.248976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 842/1658] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:05.253850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:05.463826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be347d51-cf86-4822-ab1f-5cf819a2a8c4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:05.477872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:05.479692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:05.526796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:05.526796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:05.526796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:05.526796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:05.551367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 817/1659] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:05 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:05.555446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:05.574584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7856e4-6cb2-4564-b10c-9be1a108f0b5 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:05.636647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:05.709864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:05.710821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:05.711248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:05.711665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:05.713227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:05.715423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:05.746621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 843/1660] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:05 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:05.751119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:05.764761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f292afe-b782-41d1-ac28-617b5db0db81 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:05.778600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:05.779813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:05.780157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:05.780712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:05.782018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:05.784189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:05.789037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:05.835904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 818/1661] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:05.840942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:05.857448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece2cbb-d7fb-4339-9fdf-cc735f5ee5e0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:05.871603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:05.872579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:05.872872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:05.873218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] PATH_INFO: `/v3/projects/dda81b99a3c14a9fb5f44245699fde5c/users/1fc9896955d940138e0cc194e00f3782/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:05.907742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda81b99a3c14a9fb5f44245699fde5c, user_id=1fc9896955d940138e0cc194e00f3782, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:05.910622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:05.947976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 844/1662] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:05 2022] PUT /identity/v3/projects/dda81b99a3c14a9fb5f44245699fde5c/users/1fc9896955d940138e0cc194e00f3782/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:05.952454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:05.967217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df7138-df36-487c-8ad9-d1e8bf998091 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:06.098932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:06.099977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:06.100579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:06.100911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:06.101653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:06.103461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fd567-279c-4e12-b8aa-e770bd78e4c8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:06.122489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:06.128366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 819/1663] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:06.136118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747087-53e1-4278-9602-74e8dd99f2fd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:06.141513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:06.143861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:06.144205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:06.145032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] PATH_INFO: `/v3/projects/dda81b99a3c14a9fb5f44245699fde5c/users/1fc9896955d940138e0cc194e00f3782/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:06.149002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda81b99a3c14a9fb5f44245699fde5c, user_id=1fc9896955d940138e0cc194e00f3782, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:06.154623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:06.186870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 845/1664] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:06 2022] PUT /identity/v3/projects/dda81b99a3c14a9fb5f44245699fde5c/users/1fc9896955d940138e0cc194e00f3782/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:06.198711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:06.199012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:06.199303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:06.265786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] MFA Rules not processed for user `1fc9896955d940138e0cc194e00f3782`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:06.267042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:06.450008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 820/1665] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:06.463621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:06.465611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:06.653127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea79854-ee40-44a0-8e6d-81bc58ad96ab admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:06.662345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:06.665004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:06.667669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:06.667669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:06.690643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:06.699472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:06.735971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 846/1666] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:07.501516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:07.513689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e61a-2332-4b0a-9b95-d6f77fac435b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:07.529135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc9896955d940138e0cc194e00f3782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dda81b99a3c14a9fb5f44245699fde5c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:07.529891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:07.530737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:07.531026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:07.531978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:07.532091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:07.536849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b337f-6ad0-40c5-997e-92bcad473d98 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:07.590679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 821/1667] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:07.595548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:07.627839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a86a-8ad7-4464-86d1-674a7566b23e service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:07.641336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:07.641833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:07.642399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:07.651172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc9896955d940138e0cc194e00f3782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dda81b99a3c14a9fb5f44245699fde5c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:07.652436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:07.652973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:07.653474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:07.656401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:07.660319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:07.746530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 847/1668] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:07 2022] POST /identity/v3/projects => generated 354 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:07.753111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] MFA Rules not processed for user `cb1342ce5ad343a190bbfc9986adbf0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:07.754683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:07.756415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:07.877639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:07.930724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 822/1669] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:07.966389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5541188-4137-4db2-9869-cfc695e24910 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:07.985619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc9896955d940138e0cc194e00f3782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dda81b99a3c14a9fb5f44245699fde5c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:07.986451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:07.986961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:07.987254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:07.988174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:07.988605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:07.991127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:08.001336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:08.018501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 848/1670] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:08.117698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:08.147192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327264-58a0-489d-ad5f-49d177edcf82 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:08.157955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fc9896955d940138e0cc194e00f3782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dda81b99a3c14a9fb5f44245699fde5c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:08.159327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:08.159770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:08.160210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] PATH_INFO: `/v3/projects/3a3ff42333ff4789a50aa20ac6655a6d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:08.164741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3a3ff42333ff4789a50aa20ac6655a6d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:08.168265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:08.236023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:08.237894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:08.326721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3ff42333ff4789a50aa20ac6655a6d', 'request_id': 'req-34e8f8f1-2a2f-4ef2-a494-12d456505959'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:08.350014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3ff42333ff4789a50aa20ac6655a6d', 'request_id': 'req-34e8f8f1-2a2f-4ef2-a494-12d456505959'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:08.372445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3ff42333ff4789a50aa20ac6655a6d', 'request_id': 'req-34e8f8f1-2a2f-4ef2-a494-12d456505959'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:08.435480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] The token cache is being invalidate because project 3a3ff42333ff4789a50aa20ac6655a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:08.435849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:08.437562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-34e8f8f1-2a2f-4ef2-a494-12d456505959 tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:08.446304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 823/1671] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:08 2022] DELETE /identity/v3/projects/3a3ff42333ff4789a50aa20ac6655a6d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:08.468546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60a23-0747-43fa-9c72-f8a160e820df tempest-VolumeQuotaClassesTest-1708055534 tempest-VolumeQuotaClassesTest-1708055534-project-admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:08.718240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:08.720692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:08.723588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:08.723929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] PATH_INFO: `/v3/users/cd12200a054240e389eca80f2c16f59c` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:08.734375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd12200a054240e389eca80f2c16f59c)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:08.745478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:08.968049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd12200a054240e389eca80f2c16f59c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.015590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd12200a054240e389eca80f2c16f59c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.030875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd12200a054240e389eca80f2c16f59c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.031691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd12200a054240e389eca80f2c16f59c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.062202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd12200a054240e389eca80f2c16f59c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.076159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 849/1672] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:08 2022] DELETE /identity/v3/users/cd12200a054240e389eca80f2c16f59c => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:09.081442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:09.081904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:09.082350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:09.098420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:09.102962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:09.124682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:09.128636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:09.162261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:09.279404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182364a0-f135-4aa0-9105-58471eb1dfce admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:09.287036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:09.288395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:09.288790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:09.289203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] PATH_INFO: `/v3/users/10e68a06b740494998459da91fe1f8e7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:09.295593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e68a06b740494998459da91fe1f8e7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:09.307009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:09.318315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 824/1673] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:09.327458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:09.329932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:09.478410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e68a06b740494998459da91fe1f8e7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.492526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e68a06b740494998459da91fe1f8e7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.517489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e68a06b740494998459da91fe1f8e7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.520145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e68a06b740494998459da91fe1f8e7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.548924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d2b7d-406e-4c67-bca0-e984e8179681 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:09.558430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:09.559284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:09.560780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:09.561195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:09.563103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7d962bba-d470-4cf6-a9b2-591ecf0cfc88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e68a06b740494998459da91fe1f8e7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:09.585298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 850/1674] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:09 2022] DELETE /identity/v3/users/10e68a06b740494998459da91fe1f8e7 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:09.590924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:09.591682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78787-a9ac-42ad-b5bd-b12ef7400903 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:09.592225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78787-a9ac-42ad-b5bd-b12ef7400903 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:09.592440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:09.592976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e78787-a9ac-42ad-b5bd-b12ef7400903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:09.619357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 825/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:09.620731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-31e78787-a9ac-42ad-b5bd-b12ef7400903 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:09.622545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-31e78787-a9ac-42ad-b5bd-b12ef7400903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:09.779017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 851/1676] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:09.817639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:09.818992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:09.846854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1c9c8-7b81-40b5-9b1f-303ab4dfbd0a service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:09.884914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:09.974876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:09.975552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:09.975945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:09.976178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:10.000909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:10.002368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f96db-a984-431f-9161-2533b29d5787 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:10.020984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 826/1677] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:10.714463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fb34-f7f3-46f6-8fa0-1672062847fe None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:10.715916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fb34-f7f3-46f6-8fa0-1672062847fe None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:10.716377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8fb34-f7f3-46f6-8fa0-1672062847fe None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:10.719104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 852/1678] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:27:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:10.770561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220955-13ba-4667-a96f-4e81f0a5f77c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:10.772252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220955-13ba-4667-a96f-4e81f0a5f77c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:10.776424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb220955-13ba-4667-a96f-4e81f0a5f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:10.820226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fb220955-13ba-4667-a96f-4e81f0a5f77c None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:10.823524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fb220955-13ba-4667-a96f-4e81f0a5f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:10.971639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 827/1679] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:11.044007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.044845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.045312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.047079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 853/1680] 10.222.0.23 () {54 vars in 960 bytes} [Thu Jun 16 00:27:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:11.052562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.052903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.053192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.084259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:11.084259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.106991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:11.108381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.117695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924338e6-9e00-49fb-836e-dbd1c4232b54 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:11.136873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 828/1681] 10.222.0.23 () {58 vars in 1057 bytes} [Thu Jun 16 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:11.148976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:11.149311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:11.150604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.150901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.151231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] PATH_INFO: `/v3/users/266a74e9540049eea14ea6a46052cd0e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.158039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=266a74e9540049eea14ea6a46052cd0e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:11.161498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:11.175429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c8ca3-4e34-4689-89bc-8ace65c9c520 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:11.190384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:11.191054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.191344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.191624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.213925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:11.216418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:11.247335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:11.262886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 829/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:11.275359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266a74e9540049eea14ea6a46052cd0e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.284632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266a74e9540049eea14ea6a46052cd0e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.293615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:11.295658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.320285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266a74e9540049eea14ea6a46052cd0e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.325972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266a74e9540049eea14ea6a46052cd0e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.344094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af2371-be60-4270-bae6-df2823c859c5 service placement] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:11.417370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:11.420811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.426067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4e8ba12e-20da-4e9d-9f4b-16ca0c1f90eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266a74e9540049eea14ea6a46052cd0e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.426611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.427174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] PATH_INFO: `/v3/projects/9eeb911e348948edaccb0f977a645c39` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.432690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eeb911e348948edaccb0f977a645c39)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:11.436433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:11.444401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 854/1683] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:11 2022] DELETE /identity/v3/users/266a74e9540049eea14ea6a46052cd0e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:11.448950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.449425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.449871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.512827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:11.516429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.583435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eeb911e348948edaccb0f977a645c39', 'request_id': 'req-986bd98c-d286-4ea7-9ee0-039e0fab683d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.618996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eeb911e348948edaccb0f977a645c39', 'request_id': 'req-986bd98c-d286-4ea7-9ee0-039e0fab683d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.640438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eeb911e348948edaccb0f977a645c39', 'request_id': 'req-986bd98c-d286-4ea7-9ee0-039e0fab683d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.681759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] The token cache is being invalidate because project 9eeb911e348948edaccb0f977a645c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:11.682072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.685021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-986bd98c-d286-4ea7-9ee0-039e0fab683d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:11.691170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 830/1684] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:11 2022] DELETE /identity/v3/projects/9eeb911e348948edaccb0f977a645c39 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:11.697559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109607-0273-4692-bdd1-19b807075578 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.697559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109607-0273-4692-bdd1-19b807075578 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.697806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8109607-0273-4692-bdd1-19b807075578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.739870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d8109607-0273-4692-bdd1-19b807075578 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:11.740355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 855/1685] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:11.742100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d8109607-0273-4692-bdd1-19b807075578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.778771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:11.876385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:11.878435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.892417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 831/1686] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:11.900873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:11.901218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:11.901507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:11.948422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:11.952372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:11.980716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 832/1687] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:11.982208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.005758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ff2d8-fa43-41e9-aa7a-bdc3d37ccae7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.019865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.021973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.022695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.023231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.025782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:12.026136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.029666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc1abe-ff76-40fb-b2f1-c077821c6429 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.031477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.073737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 833/1688] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:27:12.075493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.115343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb76916-f270-4275-8e5a-5ff6ae984c6b admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.130613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.131862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.131862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.132075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.133848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.143333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.192104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.198455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.198791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.199081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.218162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 834/1689] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:12 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:12.220261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.230089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.235624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.265848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 856/1690] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:12.376384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8121e-4f22-49b5-87b4-f9d08bfeeb43 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.386739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.387479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.387772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.388059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.388797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.390851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.451137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 835/1691] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:12 2022] POST /identity/v3/users => generated 284 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:12.452133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.505529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451604e-8e39-42a0-9ef5-548684edd9ce service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.513715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.514507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.514810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.515087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.515892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.517817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.529125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:12.536518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 857/1692] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:12.541744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.560990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a213aec-72c5-4766-8a6c-941a6409164d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.570700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.573102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.574151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.575961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.576824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.580385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.611323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.613470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:12.613616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 836/1693] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:12.614540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.637551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb59ac0-b88d-4c82-bed6-764e58651387 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.640512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acde0cb-0008-42b6-87a8-ffc346212149 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:12.651799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.653136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.653430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.653624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:12.653740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] PATH_INFO: `/v3/projects/0c32911b5f104dcba4cad89bfc78769d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.655806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.657389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c32911b5f104dcba4cad89bfc78769d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.658452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: WARNING py.warnings [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 00:27:12.658452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: warnings.warn(msg) Jun 16 00:27:12.658452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: Jun 16 00:27:12.658710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.659669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.661153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] PATH_INFO: `/v3/projects/922dbc3f2f6b4a13a7fe3ca14807571d/users/a38d2ed38b504fdb8e1c723d975cbfe0/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.718263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=922dbc3f2f6b4a13a7fe3ca14807571d, user_id=a38d2ed38b504fdb8e1c723d975cbfe0, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:12.737159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4b5fc-481f-40e8-8004-2f1a8cb0e079 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:12.791576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 837/1694] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:12 2022] PUT /identity/v3/projects/922dbc3f2f6b4a13a7fe3ca14807571d/users/a38d2ed38b504fdb8e1c723d975cbfe0/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:12.801045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c32911b5f104dcba4cad89bfc78769d', 'request_id': 'req-4cf6de94-45cc-40db-a8da-cef42fa7fadf'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:12.805407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18f632-402e-4a1b-b7aa-d466ce979373 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:12.812064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18f632-402e-4a1b-b7aa-d466ce979373 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:12.812914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18f632-402e-4a1b-b7aa-d466ce979373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:12.832829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c32911b5f104dcba4cad89bfc78769d', 'request_id': 'req-4cf6de94-45cc-40db-a8da-cef42fa7fadf'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:12.847596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c32911b5f104dcba4cad89bfc78769d', 'request_id': 'req-4cf6de94-45cc-40db-a8da-cef42fa7fadf'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:12.883277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-af18f632-402e-4a1b-b7aa-d466ce979373 None None] MFA Rules not processed for user `a38d2ed38b504fdb8e1c723d975cbfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:12.886652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-af18f632-402e-4a1b-b7aa-d466ce979373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:12.894817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] The token cache is being invalidate because project 0c32911b5f104dcba4cad89bfc78769d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:12.895122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:12.900614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:12.903103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 858/1695] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:12 2022] DELETE /identity/v3/projects/0c32911b5f104dcba4cad89bfc78769d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:12.906886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:12.908402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:12.940720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:13.066555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 838/1696] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:13.070636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.070939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.071229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.130123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6de94-45cc-40db-a8da-cef42fa7fadf admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.157086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] MFA Rules not processed for user `01c74d394beb4d7286563fc4a9687d50`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:13.160011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.294436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.295369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.295668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.295953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] PATH_INFO: `/v3/users/1fc9896955d940138e0cc194e00f3782` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.297289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc9896955d940138e0cc194e00f3782)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:13.299619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:13.359401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 839/1697] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:13.361823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:13.364445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.370232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:13.412441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e072679-ed5b-4d1e-b149-22964a020313 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.424974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc9896955d940138e0cc194e00f3782'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:13.427055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.428349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.432934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.433514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.436469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:13.441704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194ec13-e14a-4d6b-b402-8d4b341befb2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:13.442761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc9896955d940138e0cc194e00f3782'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:13.472632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc9896955d940138e0cc194e00f3782'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:13.475262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc9896955d940138e0cc194e00f3782'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:13.514347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 840/1698] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:13 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:13.515929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.516609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.517115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.526899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc9896955d940138e0cc194e00f3782'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:13.545020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 859/1699] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:12 2022] DELETE /identity/v3/users/1fc9896955d940138e0cc194e00f3782 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:13.546002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:13.548152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] MFA Rules not processed for user `01c74d394beb4d7286563fc4a9687d50`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:13.549997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.551597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.647452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 841/1700] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:13.660223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:13.661755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.732612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef6efe-a80c-4816-aae3-76e9bdf2fc9f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.748678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.749554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.752227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.752577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.754554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:13.761277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:13.770475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:13.807090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 860/1701] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:13 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:13.811952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:13.844335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfdb46-7a34-4bb5-96c1-d72faafc9c4f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.858445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.859209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.859505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.859789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.860601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:13.865215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:13.881237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af75921-a279-4cff-8ae4-9b962c769e1a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.898504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.898795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 861/1702] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:13.899205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.899778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:13.899981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.905616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.909126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:13.925515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f51f32-6aee-43c1-828c-466696d1c2b6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:13.937882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:13.939594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:13.944065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:13.947158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:13.948003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:13.948434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] PATH_INFO: `/v3/projects/6881fd8939cf46c7a72789faca5b6399/users/acd81c01063b405998c4f26cbea74743/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:13.960767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 842/1703] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:14.004191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] RBAC: Authorizing `identity:create_grant(project_id=6881fd8939cf46c7a72789faca5b6399, user_id=acd81c01063b405998c4f26cbea74743, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:14.007499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:14.050637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:14.051022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 862/1704] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:13 2022] PUT /identity/v3/projects/6881fd8939cf46c7a72789faca5b6399/users/acd81c01063b405998c4f26cbea74743/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:14.066424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a003a0-3f6d-4609-a59e-469dd6a02757 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:14.142912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:14.144977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:14.208950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:14.209660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.210419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.210419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.211055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:14.215066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:14.247194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 843/1705] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:14.248358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:14.261637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd1bf-2395-44f9-b988-fac0ad3cee55 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:14.270175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:14.272172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.273508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.275037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] PATH_INFO: `/v3/projects/6881fd8939cf46c7a72789faca5b6399/users/acd81c01063b405998c4f26cbea74743/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.282325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6881fd8939cf46c7a72789faca5b6399, user_id=acd81c01063b405998c4f26cbea74743, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:14.290201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:14.302949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d9790-9d2d-4754-928d-9aa1f527f5da admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:14.314835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:14.315743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.316041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.316316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] PATH_INFO: `/v3/users/097d0b953daf4d75b2cbf94f9324915c` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.318668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 844/1706] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:14 2022] PUT /identity/v3/projects/6881fd8939cf46c7a72789faca5b6399/users/acd81c01063b405998c4f26cbea74743/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:14.319654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:14.321930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=097d0b953daf4d75b2cbf94f9324915c)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:14.325058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:14.327624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:14.348753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:14.451993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097d0b953daf4d75b2cbf94f9324915c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.469305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097d0b953daf4d75b2cbf94f9324915c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.487094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097d0b953daf4d75b2cbf94f9324915c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.489753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097d0b953daf4d75b2cbf94f9324915c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.531422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce53d-55cd-4214-9757-61bec5f5fc04 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:14.536173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e250933-0f40-4368-a9e4-609bfaed68ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097d0b953daf4d75b2cbf94f9324915c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.545989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:14.549444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.549750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.550178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] PATH_INFO: `/v3/projects/8256b9c4822941639662d00143e57bf4` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.556096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 863/1707] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:14 2022] DELETE /identity/v3/users/097d0b953daf4d75b2cbf94f9324915c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:14.557829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8256b9c4822941639662d00143e57bf4)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:14.559828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:14.560121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997dfa-b0e6-4aa1-9821-2f5b230fa69c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.560432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997dfa-b0e6-4aa1-9821-2f5b230fa69c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.561484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85997dfa-b0e6-4aa1-9821-2f5b230fa69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.630176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-85997dfa-b0e6-4aa1-9821-2f5b230fa69c None None] MFA Rules not processed for user `acd81c01063b405998c4f26cbea74743`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:14.636240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-85997dfa-b0e6-4aa1-9821-2f5b230fa69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:14.684966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:14.692788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8256b9c4822941639662d00143e57bf4', 'request_id': 'req-6efe0525-a2ec-4654-b8c1-efda1fa45341'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.709570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8256b9c4822941639662d00143e57bf4', 'request_id': 'req-6efe0525-a2ec-4654-b8c1-efda1fa45341'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.731658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8256b9c4822941639662d00143e57bf4', 'request_id': 'req-6efe0525-a2ec-4654-b8c1-efda1fa45341'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.786958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] The token cache is being invalidate because project 8256b9c4822941639662d00143e57bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:14.789085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.792812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:14.795760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 845/1708] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:14 2022] DELETE /identity/v3/projects/8256b9c4822941639662d00143e57bf4 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:14.796790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:14.801898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:14.911113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 864/1709] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:14.911113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6b7c-eff5-4fa8-8ab7-a7ce8a3fea72 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:14.911113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6b7c-eff5-4fa8-8ab7-a7ce8a3fea72 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:14.926040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6b7c-eff5-4fa8-8ab7-a7ce8a3fea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:14.971222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b84b6b7c-eff5-4fa8-8ab7-a7ce8a3fea72 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:14.972506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b84b6b7c-eff5-4fa8-8ab7-a7ce8a3fea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.023208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe0525-a2ec-4654-b8c1-efda1fa45341 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:15.045239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:15.046162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.051426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.052314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] PATH_INFO: `/v3/users/193aeba5796440c7bda593fb719ec0d7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.056444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=193aeba5796440c7bda593fb719ec0d7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:15.058554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:15.195305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193aeba5796440c7bda593fb719ec0d7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:15.202861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 865/1710] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:15.204612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.205142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.205620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.221240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193aeba5796440c7bda593fb719ec0d7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:15.255169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:15.257291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.263964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193aeba5796440c7bda593fb719ec0d7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:15.264711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193aeba5796440c7bda593fb719ec0d7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:15.267132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193aeba5796440c7bda593fb719ec0d7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:15.270894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 866/1711] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:15.272076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:15.276272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.288916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:15.302712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 846/1712] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:14 2022] DELETE /identity/v3/users/193aeba5796440c7bda593fb719ec0d7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:15.302712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:15.306726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.421920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf7535-e372-4e05-bc72-426b78383c1d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:15.503940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f916-dcbd-4ec5-b882-400347ed7c93 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:15.514269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:15.514974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.515271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.515564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.520244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.535873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:15.609689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:15.610436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.610732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.611020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.615481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.714462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:15.717456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b438fe-6a54-439b-8505-8abcc27c095b service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:15.754076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 847/1713] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:15.759174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc454-b838-4706-a5ea-8bc0dc8466bb None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.759465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc454-b838-4706-a5ea-8bc0dc8466bb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.759749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dc454-b838-4706-a5ea-8bc0dc8466bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.809191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:15.811366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3aa2fd-aceb-4afb-bf85-a77ec7d8ce81 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:15.826078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-556dc454-b838-4706-a5ea-8bc0dc8466bb None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:15.827498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-556dc454-b838-4706-a5ea-8bc0dc8466bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.834611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 867/1714] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:15.836405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.837896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.838186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.841095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 868/1715] 10.222.0.23 () {54 vars in 912 bytes} [Thu Jun 16 00:27:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:15.841869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:15.850750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 848/1716] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:15.851859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.853832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.854798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.855096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.915911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854222e-27ff-4a21-aaa4-08c774c85664 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:15.935651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:15.936575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:15.936867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:15.937175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] PATH_INFO: `/v3/projects/dda81b99a3c14a9fb5f44245699fde5c` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:15.939870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda81b99a3c14a9fb5f44245699fde5c)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:15.946490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:15.972391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:15.973772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:15.986516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 849/1717] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:15.987615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:16.041289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19468b2b-d05a-48dd-91f9-a7e9333359ca None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:16.070487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda81b99a3c14a9fb5f44245699fde5c', 'request_id': 'req-6e3d0210-52c2-4842-a221-526bae8dedca'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:16.085415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda81b99a3c14a9fb5f44245699fde5c', 'request_id': 'req-6e3d0210-52c2-4842-a221-526bae8dedca'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:16.094872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:16.107261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda81b99a3c14a9fb5f44245699fde5c', 'request_id': 'req-6e3d0210-52c2-4842-a221-526bae8dedca'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:16.137816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] The token cache is being invalidate because project dda81b99a3c14a9fb5f44245699fde5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:16.138614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:16.141310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:16.147387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 869/1718] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:15 2022] DELETE /identity/v3/projects/dda81b99a3c14a9fb5f44245699fde5c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:16.150835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:16.152945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.205899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:16.209619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.210030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.210439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.211306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:16.213560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:16.218321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:16.240816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 850/1719] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:16.241854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:16.251156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.288264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d0210-52c2-4842-a221-526bae8dedca admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:16.298748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:16.299659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.300051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.300436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.307498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.370849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff79bc-7421-4606-b14d-bc80ff5d3a87 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:16.374821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:16.375567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.375799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.376179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.378602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.399857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:16.401918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b8ef1-ce74-4542-9128-af4fcaed2079 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:16.433693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 851/1720] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:16.435238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ca7c-ec61-4799-aa66-a867517e8b97 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.435541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ca7c-ec61-4799-aa66-a867517e8b97 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.436191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ca7c-ec61-4799-aa66-a867517e8b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.452174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:16.459294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0ed6e-14e2-4d8b-bf5a-8d9bdbe97d82 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:16.487339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:16.488357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 870/1721] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:16.492005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.492909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.493346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.493684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-78e2ca7c-ec61-4799-aa66-a867517e8b97 None None] MFA Rules not processed for user `f1d7bb8f9d364c5198313fedd982478a`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:16.494847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-78e2ca7c-ec61-4799-aa66-a867517e8b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.556916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:16.558546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.593384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 871/1722] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:16.599094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:16.601734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.631679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d9d89-60f8-4251-b4ac-2d4ebb7f3a43 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:16.646917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:16.648522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.648522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.648522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.649422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:16.654288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:16.714347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 852/1723] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:16.718766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.719062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.719345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.750287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 872/1724] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:16 2022] POST /identity/v3/projects => generated 405 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:16.750287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:16.751660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.787890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:16.789196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:16.964106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0decbf6-f03e-445a-80eb-fb50a1e5366c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:16.983579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:16.986028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:16.986345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:16.986665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] PATH_INFO: `/v3/projects/3bf50ba5c25c4892ae4ecea7f5346e65` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:16.992018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf50ba5c25c4892ae4ecea7f5346e65)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:16.997337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.005454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 853/1725] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:17.006412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.008345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.066011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:17.145445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf50ba5c25c4892ae4ecea7f5346e65', 'request_id': 'req-7111e1c3-af42-4f39-a491-15c188b741cd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.170937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf50ba5c25c4892ae4ecea7f5346e65', 'request_id': 'req-7111e1c3-af42-4f39-a491-15c188b741cd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.192302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf50ba5c25c4892ae4ecea7f5346e65', 'request_id': 'req-7111e1c3-af42-4f39-a491-15c188b741cd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.231908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] The token cache is being invalidate because project 3bf50ba5c25c4892ae4ecea7f5346e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:17.232252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.239076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.240815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 873/1726] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:16 2022] DELETE /identity/v3/projects/3bf50ba5c25c4892ae4ecea7f5346e65 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:17.245912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.247953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.249473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238edbce-8844-4db6-b408-bf9f65261151 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.340966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:17.393621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.395542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.395841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.397413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.401976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.412989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e1c3-af42-4f39-a491-15c188b741cd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.425013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.427764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.428066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.428684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.431605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.434961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.477474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 874/1727] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:17 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:17.478347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.480289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.499275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1bb20-1f93-496d-a87f-15240b529e61 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.507804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.508507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.509392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.509677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.510585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:17.510904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.512527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.532544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 875/1728] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:17.533556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.535295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.549973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538b488-5e21-488a-944a-bdbe252755d8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.556198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.557374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.557665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.557947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] PATH_INFO: `/v3/projects/e6364c3ea3274ff690cd68f68f215784` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.559186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6364c3ea3274ff690cd68f68f215784)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.561110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.566052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.568712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.603689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 854/1729] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:17.604561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.606057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.623757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f914891-3cd7-46db-8ed1-d9ec7281a749 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.631215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.631787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.632079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.632354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] PATH_INFO: `/v3/users/cb1342ce5ad343a190bbfc9986adbf0b` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.633612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1342ce5ad343a190bbfc9986adbf0b)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.635578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.670733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6364c3ea3274ff690cd68f68f215784', 'request_id': 'req-91f9f204-1462-4934-9714-571f39e2c34f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.707246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6364c3ea3274ff690cd68f68f215784', 'request_id': 'req-91f9f204-1462-4934-9714-571f39e2c34f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.710801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1342ce5ad343a190bbfc9986adbf0b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.730470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6364c3ea3274ff690cd68f68f215784', 'request_id': 'req-91f9f204-1462-4934-9714-571f39e2c34f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.730656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1342ce5ad343a190bbfc9986adbf0b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.746424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] The token cache is being invalidate because project e6364c3ea3274ff690cd68f68f215784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:17.747262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.748440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91f9f204-1462-4934-9714-571f39e2c34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.754224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 876/1730] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:17 2022] DELETE /identity/v3/projects/e6364c3ea3274ff690cd68f68f215784 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:17.756825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.758241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.758923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.766246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1342ce5ad343a190bbfc9986adbf0b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.767060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1342ce5ad343a190bbfc9986adbf0b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.768323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1342ce5ad343a190bbfc9986adbf0b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:17.776019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:17.794409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] MFA Rules not processed for user `b5f4125ecaec41e5ba5c4eaf00c43e07`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:17.796174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.800429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 855/1731] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:17 2022] DELETE /identity/v3/users/cb1342ce5ad343a190bbfc9986adbf0b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:17.801518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.810459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.888523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 877/1732] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:17.889586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.890994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:17.919007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:17.948261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28214-ec5d-4614-ae4c-4849f047b8be admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:17.956954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:17.957875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:17.959462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:17.959754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:17.961555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:17.963524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:17.984007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 856/1733] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:17.985496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:17.988404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.020561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f6ddb-070a-4e7a-818a-4d68216f4e6a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:18.028015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:18.029648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.030635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.031092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.032471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:18.037081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ee7bc-c8e8-4d0a-abb6-700ab47f8543 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:18.098824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 878/1734] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:17 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:18.103669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.103969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.104257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.108079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404e220-ac54-4f56-b804-0a24dabcb1c0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:18.163227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:18.164258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.264068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:18.265486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.266187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.266187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.269692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.314324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:18.316277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719136c-313d-4251-bb20-c85778adb815 service placement] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:18.330913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 857/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:18.333918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.334236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.334508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.348907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 879/1736] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:18.349865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:18.356921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] MFA Rules not processed for user `b5f4125ecaec41e5ba5c4eaf00c43e07`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:18.357841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.385458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bea773-c8f6-492b-b904-c6dbdb603b4b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:18.401655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:18.402363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.402648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.402927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.403674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:18.405338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:18.455908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 880/1737] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:18.462952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:18.465206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 858/1738] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:18.467146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:18.468522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.488229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332bb0f-1cef-4e3f-a6d2-78db1821c834 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:18.496398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:18.497875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.498380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.498885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] PATH_INFO: `/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.501451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8509e01e3fb497c8a705c003e0c7c4d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:18.505444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:18.510457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f7d1e-ad2e-4cc0-8c93-b768ed0d2a2d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:18.556996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:18.557763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.557981 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.558260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.567346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:18.590503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820419f2-da46-40c2-a694-4e89df851a1a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:18.593532 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:18.631992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:18.685029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 881/1739] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:18 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:18.688583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:18.688898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:18.690513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:18.714183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8509e01e3fb497c8a705c003e0c7c4d', 'request_id': 'req-dddc9a74-8153-4d87-a632-455626ec7e63'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:18.769393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8509e01e3fb497c8a705c003e0c7c4d', 'request_id': 'req-dddc9a74-8153-4d87-a632-455626ec7e63'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:18.777835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:18.779034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:18.805346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8509e01e3fb497c8a705c003e0c7c4d', 'request_id': 'req-dddc9a74-8153-4d87-a632-455626ec7e63'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:18.854069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] The token cache is being invalidate because project a8509e01e3fb497c8a705c003e0c7c4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:18.873459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:18.874759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:18.878080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 859/1740] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:18 2022] DELETE /identity/v3/projects/a8509e01e3fb497c8a705c003e0c7c4d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:18.889707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:18.907421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.025135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 882/1741] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:19.027670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:19.034898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.157655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc9a74-8153-4d87-a632-455626ec7e63 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:19.172405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:19.177248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.182140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.183115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] PATH_INFO: `/v3/projects/c328294bb9f944ecabb83e3399646535/users/be12a33b86e0485e917fb6859e18981a/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.244483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c328294bb9f944ecabb83e3399646535, user_id=be12a33b86e0485e917fb6859e18981a, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:19.255292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:19.304520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579f2df-e81f-41cf-aecb-148fed56a3b9 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:19.315973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 860/1742] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:18 2022] PUT /identity/v3/projects/c328294bb9f944ecabb83e3399646535/users/be12a33b86e0485e917fb6859e18981a/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:19.327223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:19.327223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.567908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d36995-3dc7-40d0-b919-3f6bbcfa3095 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:19.569702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:19.570396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.570665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.570944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.577809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.581943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:19.582703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.583002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.583379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.584222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:19.586055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d106234-c962-4b37-8fdb-b63d7962fb1d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:19.625311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 861/1743] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:19.630053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.631775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.632275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.679384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:19.686495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:19.693549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.723287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 862/1744] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:19.734468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:19.737545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:19.739197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0110594-5630-40bc-a0e5-955a19d233bc service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:19.763671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 883/1745] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:19.765161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.765466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.765811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.768037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ccb22-9806-4aa3-ae5f-33c0d6050b3a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:19.785149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:19.786023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.786466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.786848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.789211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:19.789678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:19.796796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:19.831654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] MFA Rules not processed for user `be12a33b86e0485e917fb6859e18981a`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:19.833304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:19.838079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 863/1746] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:19.839846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:19.873338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1798f-6cc3-44f3-9ec9-6efd927736d0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:19.903120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:19.904066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.904427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.904964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:19.906055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:19.907767 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867b680-43d1-4783-82fc-007f09f74b9a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:19.912635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:19.950219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 864/1747] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:19.955395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:19.955807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:19.956166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.010015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:20.011714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:20.028851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 884/1748] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:20.030375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.037025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 865/1749] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:20.043250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.052160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757c25-46de-4d44-8531-e34e57edcdb7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.067950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515fb05-3288-4670-8ff7-bd67fc3b407a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.075856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.077067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.077563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.078063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.078984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.079428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.081247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.082996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.083443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] PATH_INFO: `/v3/projects/cdb09585bf494540b8f77af8a84a3227/users/4bdd28b9e58742b082d26d4107e00323/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.083773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.109181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb09585bf494540b8f77af8a84a3227, user_id=4bdd28b9e58742b082d26d4107e00323, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.112506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.136872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 866/1750] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:20 2022] PUT /identity/v3/projects/cdb09585bf494540b8f77af8a84a3227/users/4bdd28b9e58742b082d26d4107e00323/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:20.138561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.148539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 885/1751] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:20.150002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.246773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bd8cf-88bc-41f1-b1df-8b78cf013e90 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.377591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.378466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.378758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.379037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.380744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.384392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9508220-a137-4a57-a8e2-a8c20cb8ee9a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.439580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e17e14-8efc-4c24-9be0-f7c7bda496f6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.460328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.460968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.461320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.464913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.465927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:20.466491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.471346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 867/1752] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/projects => generated 405 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:20.478268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.478600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.478896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.482803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.524677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 886/1753] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:20.525697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.566457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] MFA Rules not processed for user `4bdd28b9e58742b082d26d4107e00323`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:20.566903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21960fce-29e9-46ff-bd30-7e6c3c1a6028 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.567738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:20.666443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:20.766743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.773629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.773876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.786323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:20.789724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.793218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.797408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.840205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 887/1754] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:20.841155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.868386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 868/1755] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:20.869341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.876025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4547-15a6-4594-8fe1-935189298130 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.889901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33ebc3-0c5d-4ed5-9eb6-1ee39f157ef2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:20.900926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.902282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.902282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.902476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.903090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.903352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:20.904039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:20.904337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:20.904633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:20.905379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:20.907451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.915630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:20.965268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 869/1756] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:20.966160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:20.994460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 888/1757] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:20.995618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.035244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aa0c5-3a34-491d-b68e-f9e39ba8f770 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.114221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86bb4-c6ed-4b90-89b6-3d9de6755e38 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.128431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.128975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.129266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.129564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.130325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.131982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.161046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 870/1758] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:21.161994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.181333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ab26-a962-469c-9b58-e9ca35640227 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.195541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.196846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.197390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.197527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.198145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.200107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.232746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.235225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.235225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.235719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.240005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.240159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 871/1759] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:21 2022] POST /identity/v3/users => generated 290 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:21.240159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.241194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.264994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b763c66-205e-4a4e-8234-7c49c2fa988f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.270921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.271628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.271919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.272340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.273403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.275058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.305814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 872/1760] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:21.306407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.326009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975139e-c844-4ede-9f44-34e045495298 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.329342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 889/1761] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:20 2022] POST /identity/v3/projects => generated 393 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:21.330670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.467150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.468192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.468611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.469083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.470044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.473633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.487415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 873/1762] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:21.488871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.501982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e7a7-5ffa-4c9d-8b93-7af5142972f7 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.509183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.510401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.510796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.511218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] PATH_INFO: `/v3/projects/39cf0716d4d644869fb895ba9306c124/users/2ea991ff25c441faa7dc0421603cfb77/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.532593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cf0716d4d644869fb895ba9306c124, user_id=2ea991ff25c441faa7dc0421603cfb77, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.533295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81401f09-8a5c-4ba6-92c7-f1fbe676e5c2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.535964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.547489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.548193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.548484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.548766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.550073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.551207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.566670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 874/1763] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:21 2022] PUT /identity/v3/projects/39cf0716d4d644869fb895ba9306c124/users/2ea991ff25c441faa7dc0421603cfb77/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:21.569105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.570404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 890/1764] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:21.571471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:21.586750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:21.591459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca308dd-d104-46b8-9687-07dcb499a5f8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.648973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:21.822998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb9c01-e847-41a9-bc9f-af4c255ee8dd admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:21.846436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:21.849461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:21.849874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:21.850469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:21.852735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:21.906056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:21.998103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 875/1765] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:21 2022] POST /identity/v3/users => generated 303 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:21.999310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.000091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.000212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.000573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.000863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] PATH_INFO: `/v3/projects/9ce2e059912949539a6dbc6dfa9971e0/users/0dd5979c1b2a4b75bf0e5afb5faf2026/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.014812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65ce2-3d42-4ce4-a933-10f4693e8fb3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.025237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.026722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.027017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.027300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.028369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.030809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.045534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce2e059912949539a6dbc6dfa9971e0, user_id=0dd5979c1b2a4b75bf0e5afb5faf2026, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.048243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fd213-8650-4c82-80c2-aaf95c5b3665 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.049439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 876/1766] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:22.050411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.065021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd4162-1665-49fc-b498-157398ed0b5e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.085575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.086271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.086599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.090816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 891/1767] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:21 2022] PUT /identity/v3/projects/9ce2e059912949539a6dbc6dfa9971e0/users/0dd5979c1b2a4b75bf0e5afb5faf2026/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:22.091052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.092373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.092669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.092982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.098404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.100088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.115693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] MFA Rules not processed for user `b5f4125ecaec41e5ba5c4eaf00c43e07`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:22.125182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.148509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 877/1768] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:22.151161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.156864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.205669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933d2d-a56b-459e-833d-c7e50990a6e2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.315514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 892/1769] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:22.316690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.379375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.380056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.380391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.380683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] PATH_INFO: `/v3/users/a38d2ed38b504fdb8e1c723d975cbfe0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.383106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a38d2ed38b504fdb8e1c723d975cbfe0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.386030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.510302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea7482-ed70-4616-b7bb-680a63fe8f0e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.515569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.516249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.516249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.516544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.518017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.519690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.530625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a38d2ed38b504fdb8e1c723d975cbfe0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:22.534864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 893/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:22.535755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.548077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ae9cb-c629-486c-9897-ea2529ef20f1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.554019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.555228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.555717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.556247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] PATH_INFO: `/v3/projects/39cf0716d4d644869fb895ba9306c124/users/2ea991ff25c441faa7dc0421603cfb77/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.556411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a38d2ed38b504fdb8e1c723d975cbfe0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:22.560337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cf0716d4d644869fb895ba9306c124, user_id=2ea991ff25c441faa7dc0421603cfb77, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.563370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada272-a481-4e58-b750-e2f4c0d85494 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.578438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 894/1771] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:22 2022] PUT /identity/v3/projects/39cf0716d4d644869fb895ba9306c124/users/2ea991ff25c441faa7dc0421603cfb77/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:22.589222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09434c29-3142-42c6-a10a-f066fbab1f53 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.589521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09434c29-3142-42c6-a10a-f066fbab1f53 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.589838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09434c29-3142-42c6-a10a-f066fbab1f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.596396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a38d2ed38b504fdb8e1c723d975cbfe0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:22.597660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a38d2ed38b504fdb8e1c723d975cbfe0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:22.599091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a38d2ed38b504fdb8e1c723d975cbfe0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:22.605496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:22.619339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-09434c29-3142-42c6-a10a-f066fbab1f53 None None] MFA Rules not processed for user `0dd5979c1b2a4b75bf0e5afb5faf2026`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:22.620588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-09434c29-3142-42c6-a10a-f066fbab1f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.640631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 878/1772] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:22 2022] DELETE /identity/v3/users/a38d2ed38b504fdb8e1c723d975cbfe0 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:22.645098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.650082 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.655655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:22.719420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 895/1773] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:22.721286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.721972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.722392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.743041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:22.744501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.829947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8583f62-8a13-4e95-8886-109f1251aa39 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:22.850426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:22.851359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.851791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.852093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] PATH_INFO: `/v3/projects/c9f7856f3d1d4ced99a13d1bedd14679/users/5c5a0b753ccd440281d5e64d185e1afd/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.897086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f7856f3d1d4ced99a13d1bedd14679, user_id=5c5a0b753ccd440281d5e64d185e1afd, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:22.902081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f027e-dc60-4023-bb81-d3d9544e7921 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:22.926792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 896/1774] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:22.928751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:22.932608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:22.933219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 879/1775] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:22 2022] PUT /identity/v3/projects/c9f7856f3d1d4ced99a13d1bedd14679/users/5c5a0b753ccd440281d5e64d185e1afd/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:22.934598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48670-623c-4a9c-91df-172a263c5e51 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:22.934890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48670-623c-4a9c-91df-172a263c5e51 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:22.935175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48670-623c-4a9c-91df-172a263c5e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:22.989687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22c7a4-e135-49f3-acc2-dd48517dda9d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.009302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-98b48670-623c-4a9c-91df-172a263c5e51 None None] MFA Rules not processed for user `2ea991ff25c441faa7dc0421603cfb77`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.014718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-98b48670-623c-4a9c-91df-172a263c5e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.117477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 880/1776] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.117477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.117477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.117477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.160537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.166632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.171648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.172721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.173150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.173553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] PATH_INFO: `/v3/projects/179d0964c8864891aaa7dc72823db5d2/users/ace11b0a48b14b68984e4542b6e1073d/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.192812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 881/1777] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.194195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:23.198136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.245374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff7f5e-7cc7-4059-a623-8062c30872c7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.246619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179d0964c8864891aaa7dc72823db5d2, user_id=ace11b0a48b14b68984e4542b6e1073d, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:23.253524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b474b-0267-492a-b0fb-9e53ca8a1510 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:23.254464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.255264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.255677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.256584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.257014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:23.262081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9e0e0-8cc4-4f0d-91f2-3c8ba403bfc8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:23.308437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 882/1778] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:23.310060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.310482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.310807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 897/1779] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:22 2022] PUT /identity/v3/projects/179d0964c8864891aaa7dc72823db5d2/users/ace11b0a48b14b68984e4542b6e1073d/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:23.312128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.317339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.317844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.318078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.340455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] MFA Rules not processed for user `f1d7bb8f9d364c5198313fedd982478a`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.341721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.370085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] MFA Rules not processed for user `5c5a0b753ccd440281d5e64d185e1afd`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:23.371594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.413558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:23.430636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:23.445534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 898/1780] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.448100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:23.449567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.518184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 883/1781] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:23.520274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:23.524516 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.564106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80ceab-4d32-4beb-8166-59c675f0bc89 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.574400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.576414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.576912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.577349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.709667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:23.711715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:23.725674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6276b6-6fbe-45fc-bced-b40f6fa8d9b8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.745708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 899/1782] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:23.748029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:23.748790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.749554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.749909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.750746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.752612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:23.874487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f05750-e193-4471-8848-e58bddb08c40 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:23.880054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:23.881275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:23.881581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:23.881971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:23.954996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:23.956645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:23.969429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 884/1783] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:27:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:27:23.971292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.033043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.035022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.039876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:24.053083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 900/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:24.053986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.078715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236424c1-d1c0-4cc1-9684-ac22910301d9 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.088416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.089261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.089585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.089922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.090787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.092562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.126774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 901/1785] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:24.127652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.146636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952aa04-6fd7-4a1c-92f2-b6771d17801e service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.152084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22733f-be58-441d-a434-31227041d4a4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.161208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.161949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.162267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.162561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.162695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.162822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.162918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.163371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.163770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.164164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.170864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c00ffa-111a-458d-9cf1-1404db9dd424 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.172771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.186381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:24.224919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 885/1786] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:23 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:24.226234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.227713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.228022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.230100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 886/1787] 10.222.0.23 () {54 vars in 912 bytes} [Thu Jun 16 00:27:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:24.231668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.233661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:24.246806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 902/1788] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:24 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:24.254115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.256925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3955b7c-7019-4f2e-ad1c-ee15db0dc0a3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.405881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa46a66-2e9d-4185-83d6-e1b6ecd941f8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.429609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.431306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.431482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.431957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] PATH_INFO: `/v3/projects/179d0964c8864891aaa7dc72823db5d2/users/ace11b0a48b14b68984e4542b6e1073d/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.437570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=179d0964c8864891aaa7dc72823db5d2, user_id=ace11b0a48b14b68984e4542b6e1073d, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.446071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.464726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.471551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.471918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.472299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] PATH_INFO: `/v3/users/be12a33b86e0485e917fb6859e18981a` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.475350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be12a33b86e0485e917fb6859e18981a)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.485551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.513501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 903/1789] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:24 2022] PUT /identity/v3/projects/179d0964c8864891aaa7dc72823db5d2/users/ace11b0a48b14b68984e4542b6e1073d/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:24.523737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.621422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be12a33b86e0485e917fb6859e18981a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:24.640734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be12a33b86e0485e917fb6859e18981a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:24.683970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be12a33b86e0485e917fb6859e18981a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:24.689211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be12a33b86e0485e917fb6859e18981a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:24.690248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-87fe27c6-af15-4494-8a13-5b8f759ecfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be12a33b86e0485e917fb6859e18981a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:24.707881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c13be-ddf0-4df1-a004-7ace4ab86b9b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.716502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.717268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.717561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.717847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.718604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:24.720235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:24.729394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 887/1790] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:24 2022] DELETE /identity/v3/users/be12a33b86e0485e917fb6859e18981a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:24.731496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ce3c4-930e-4e6f-9254-fef0000d67fa None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.731941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ce3c4-930e-4e6f-9254-fef0000d67fa None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.732435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ce3c4-930e-4e6f-9254-fef0000d67fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.775517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 904/1791] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:24.780154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:24.781503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-955ce3c4-930e-4e6f-9254-fef0000d67fa None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:24.782942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-955ce3c4-930e-4e6f-9254-fef0000d67fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:24.788022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:24.847880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdedab83-cc8e-447f-8baf-0e5891cd621d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:24.951100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 888/1792] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:24.952345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:24.952636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:24.952940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:24.972951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:24.997456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:24.999733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.000177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.000462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.002282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.006900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc43a2-8e11-4a3d-a6ed-c8e7c9fb749e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.031163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] MFA Rules not processed for user `ace11b0a48b14b68984e4542b6e1073d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:25.032392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.067347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:25.073673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 905/1793] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:24 2022] POST /identity/v3/users => generated 296 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:25.078335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22190b-2cb8-42c3-9bce-72b1eea330c8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.078676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22190b-2cb8-42c3-9bce-72b1eea330c8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.078960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22190b-2cb8-42c3-9bce-72b1eea330c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.150990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4d22190b-2cb8-42c3-9bce-72b1eea330c8 None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:25.158537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4d22190b-2cb8-42c3-9bce-72b1eea330c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.190663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 906/1794] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 16 00:27:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:25.192144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.192512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.195328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.254501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:25.257288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.287089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 889/1795] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:25.291348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.291488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 907/1796] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:25.291590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.298180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.305832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.306030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529709b1-18e3-4b21-b6e8-af6e613c60fd None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.310909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.311818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.312101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.312373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] PATH_INFO: `/v3/projects/3fd02b66b878489787c3bd4b333aec99/users/9207c706caca4f14a0f1f694c5954a67/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.384825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5544-791c-43c6-88a2-0022ec7fba10 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.389472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd02b66b878489787c3bd4b333aec99, user_id=9207c706caca4f14a0f1f694c5954a67, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.392478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.393376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.393989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.394825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.403721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.404624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.406387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.417149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 908/1797] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:25.418158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.444262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 890/1798] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:25 2022] PUT /identity/v3/projects/3fd02b66b878489787c3bd4b333aec99/users/9207c706caca4f14a0f1f694c5954a67/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:25.447473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9c78a-d723-4428-adb2-5253b11dc609 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.448591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.451670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.577190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.578210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.578971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.579536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.582670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.583703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.604224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 909/1799] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:25.605850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.609419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.713204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c84b54-f949-47ea-aa58-2896e542e206 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.731785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.733225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.736001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.738625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.751222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.753104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.768197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 891/1800] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:25.771828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.774932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:25.790485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181188a8-1244-4657-927a-01f69456fa28 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.806301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.819124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.822109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.822838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.876230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.879489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.910728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 910/1801] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:25.913257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:25.945403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55747f0-2b38-4ace-a6e2-4e9ae9cc229b service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:25.959942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:25.963104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:25.963104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:25.963104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:25.963104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:25.970448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:25.988564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec74b-1f8e-443e-bbc9-78a711eb316c service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.003119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.003808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.004099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.004373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.007034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 911/1802] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:26.009646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.016205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:26.048034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7b0b2-f536-4a17-b805-d8b366a28d13 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.061438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.064679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.065424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.065905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] PATH_INFO: `/v3/projects/922dbc3f2f6b4a13a7fe3ca14807571d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.069740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] RBAC: Authorizing `identity:delete_project(project_id=922dbc3f2f6b4a13a7fe3ca14807571d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.077676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.132353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:26.175945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:26.194115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922dbc3f2f6b4a13a7fe3ca14807571d', 'request_id': 'req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.206917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.207788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922dbc3f2f6b4a13a7fe3ca14807571d', 'request_id': 'req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.208418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.224637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922dbc3f2f6b4a13a7fe3ca14807571d', 'request_id': 'req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.239575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 892/1803] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:26.240937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.260873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] The token cache is being invalidate because project 922dbc3f2f6b4a13a7fe3ca14807571d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:26.261466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.264839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.267925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 912/1804] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:26 2022] DELETE /identity/v3/projects/922dbc3f2f6b4a13a7fe3ca14807571d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:26.275452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ec220-a8bd-4a38-8973-14956321eb57 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.275852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.282229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:26.412163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.413404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.413792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.414829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] PATH_INFO: `/v3/projects/f6be6498afac4a5383974417020759ff/users/9731b9935d824810b4d24c0da2e3a900/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.438836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6be6498afac4a5383974417020759ff, user_id=9731b9935d824810b4d24c0da2e3a900, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.451668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.491914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 893/1805] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:26 2022] PUT /identity/v3/projects/f6be6498afac4a5383974417020759ff/users/9731b9935d824810b4d24c0da2e3a900/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:26.491914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.495531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:26.541512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a27cd0-d1bf-44c5-b7b1-7b44bd77513d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.654734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d32138-a8b5-40d1-bbf8-86a83282c4b6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.666116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.667352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.667800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.668301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] PATH_INFO: `/v3/users/acd81c01063b405998c4f26cbea74743` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.669978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd81c01063b405998c4f26cbea74743)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.676703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.697478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:26.698555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:26.698846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:26.699947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] PATH_INFO: `/v3/projects/3fd02b66b878489787c3bd4b333aec99/users/9207c706caca4f14a0f1f694c5954a67/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:26.708075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd02b66b878489787c3bd4b333aec99, user_id=9207c706caca4f14a0f1f694c5954a67, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:26.715200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:26.748336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 913/1806] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:26 2022] PUT /identity/v3/projects/3fd02b66b878489787c3bd4b333aec99/users/9207c706caca4f14a0f1f694c5954a67/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:26.751441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.753504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:26.822106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd81c01063b405998c4f26cbea74743'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.845342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd81c01063b405998c4f26cbea74743'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.872521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:26.919002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd81c01063b405998c4f26cbea74743'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.921382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd81c01063b405998c4f26cbea74743'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.923755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd81c01063b405998c4f26cbea74743'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:26.952550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d22d5e-a0ad-4a77-b596-a2f31c84440f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:26.960164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 894/1807] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:26 2022] DELETE /identity/v3/users/acd81c01063b405998c4f26cbea74743 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:26.963387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:26.965602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:27.017600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:27.019473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:27.020414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:27.020697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:27.030086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:27.135520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:27.222865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72940e0c-2e1e-4093-9a63-c15affcdae2e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:27.247895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:27.248715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:27.249109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:27.249467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:27.260122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:27.269247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed349556-9f7b-4fa2-93d5-9368774100db admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:27.292613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:27.296547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:27.329554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 895/1808] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:27.329993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:27.330432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:27.330718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:27.347076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 914/1809] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:27.351554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:27.428228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] MFA Rules not processed for user `9207c706caca4f14a0f1f694c5954a67`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:27.431752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:27.560167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e5d48-ebf2-4204-8a00-dd82c0476617 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:27.576589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:27.577583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:27.581465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:27.581763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] PATH_INFO: `/v3/projects/f6be6498afac4a5383974417020759ff/users/9731b9935d824810b4d24c0da2e3a900/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:27.590235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6be6498afac4a5383974417020759ff, user_id=9731b9935d824810b4d24c0da2e3a900, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:27.593119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 896/1810] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:27.598258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71a9b1-5f69-4113-a928-6588f22876e0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:27.611103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:27.615235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:27.634344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 915/1811] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:27 2022] PUT /identity/v3/projects/f6be6498afac4a5383974417020759ff/users/9731b9935d824810b4d24c0da2e3a900/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:27.640773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:27.641172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:27.641428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:27.718765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] MFA Rules not processed for user `9731b9935d824810b4d24c0da2e3a900`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:27.720045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:27.882158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774139-6772-4855-8131-a50e6bd9b1ed None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:27.909391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:28.030006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 916/1812] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:28.079209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:28.079925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.080221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.080509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.134924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:28.136575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:28.167391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b884f17-2526-438a-95bc-c2d6820e9a30 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:28.183254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:28.185430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.186621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.188301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.220333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:28.223691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:28.278088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 917/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:28.407682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:28.409203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:28.470302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 897/1814] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 861 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:28.627182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:28.647918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d6095-7d8b-480a-a0f0-c611f8ef9e49 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:28.669027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:28.673814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.674259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.674370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.694605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:28.848712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:28.855697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:28.868215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea037e50-61c6-494b-bb88-e4db5bd87b2a service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:28.911583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:28.912262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:28.922155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:28.922462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:28.925962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:28.952927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:28.959049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:28.990296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 918/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:29.061773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:29.064507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:29.089906 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 898/1816] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:29.630508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:29.634801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:29.872171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7b9c9-8b71-4016-b161-643fc0dbf2f3 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:29.886526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:29.902351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:29.906911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:29.907154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] PATH_INFO: `/v3/projects/c328294bb9f944ecabb83e3399646535` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:29.910418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c328294bb9f944ecabb83e3399646535)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:29.920732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:30.126701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c328294bb9f944ecabb83e3399646535', 'request_id': 'req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.155118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c328294bb9f944ecabb83e3399646535', 'request_id': 'req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.167426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c328294bb9f944ecabb83e3399646535', 'request_id': 'req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.190930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] The token cache is being invalidate because project c328294bb9f944ecabb83e3399646535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:30.191223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.197216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.199299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 919/1817] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:29 2022] DELETE /identity/v3/projects/c328294bb9f944ecabb83e3399646535 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:30.203528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:30.231568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81c86d-2f1c-4235-9bb5-c2787856ea02 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:30.352322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:30.354504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:30.531759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:30.538740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:30.541830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:30.542938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] PATH_INFO: `/v3/users/2ea991ff25c441faa7dc0421603cfb77` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:30.548394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea991ff25c441faa7dc0421603cfb77)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:30.567923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:30.581649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:30.590882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec8c5-3fd8-48a9-9395-e1a4195f79f8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:30.626156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:30.627404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:30.630865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:30.631580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] PATH_INFO: `/v3/projects/6881fd8939cf46c7a72789faca5b6399` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:30.634861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6881fd8939cf46c7a72789faca5b6399)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:30.640482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:30.642764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:30.792120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea991ff25c441faa7dc0421603cfb77'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.823124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea991ff25c441faa7dc0421603cfb77'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.849526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea991ff25c441faa7dc0421603cfb77'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.853695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea991ff25c441faa7dc0421603cfb77'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.854865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8d3d3719-2408-4310-b419-7d15b65665fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea991ff25c441faa7dc0421603cfb77'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.890687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6881fd8939cf46c7a72789faca5b6399', 'request_id': 'req-080435f3-2058-4279-b80c-d0d03ef55b93'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.899301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 899/1818] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:30 2022] DELETE /identity/v3/users/2ea991ff25c441faa7dc0421603cfb77 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:30.900977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e4c0-0ae7-4e3b-a51b-1fb8b56d0130 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:30.901282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e4c0-0ae7-4e3b-a51b-1fb8b56d0130 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:30.901568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7e4c0-0ae7-4e3b-a51b-1fb8b56d0130 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:30.905982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6881fd8939cf46c7a72789faca5b6399', 'request_id': 'req-080435f3-2058-4279-b80c-d0d03ef55b93'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.907944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 900/1819] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:27:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:30.937568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6881fd8939cf46c7a72789faca5b6399', 'request_id': 'req-080435f3-2058-4279-b80c-d0d03ef55b93'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.959341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] The token cache is being invalidate because project 6881fd8939cf46c7a72789faca5b6399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:30.960226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.961276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-080435f3-2058-4279-b80c-d0d03ef55b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:30.966856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 920/1820] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:30 2022] DELETE /identity/v3/projects/6881fd8939cf46c7a72789faca5b6399 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:30.972176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42aa1e-669d-472a-a3c7-868248a2d249 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:30.972477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42aa1e-669d-472a-a3c7-868248a2d249 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:30.972762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42aa1e-669d-472a-a3c7-868248a2d249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.000117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.002092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.005350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.018147 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-be42aa1e-669d-472a-a3c7-868248a2d249 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:31.019336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-be42aa1e-669d-472a-a3c7-868248a2d249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.125352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:31.137791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.251475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 901/1821] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:31.257694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.259464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.259916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.309731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:31.314928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.347040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 902/1822] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:31.351185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:31.387881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f0b06-ac2d-4b39-a7c2-55f404f79bd5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:31.406817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:31.413628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.414053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.414940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.420180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:31.420710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:31.428072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:31.494754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 903/1823] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:31.496129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:31.511904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 921/1824] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:31.543083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8d811-06d7-4080-a38a-9e44237d5d74 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:31.557481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:31.558257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.558508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.558789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.559580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:31.561737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:31.653958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 904/1825] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:31.658425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:31.686279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56a26-94b4-44ce-b0dd-49f55d430d82 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:31.809491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:31.811909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:31.945776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:31.946898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:31.947344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:31.947777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:31.948803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:31.954997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:31.957293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:31.999377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 922/1826] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:31 2022] POST /identity/v3/users => generated 290 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:32.003364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:32.030581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062f631-9513-470c-8b4b-1e816767b8b1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.042177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d58f40-b9cd-4a9d-a862-33e76be5d61b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.043486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.044215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.044501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.044855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.065709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.065901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.066718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.067356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.067480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.067890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.069892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.072698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.079220 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:32.089992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 923/1827] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:32.091991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 905/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:32.108230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:32.134136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222c6b8-65fc-4416-873e-e2d405763c52 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.146654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.147552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.148098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.148387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.149807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.151480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cb800-b6bd-48b4-b48a-00967a0b4947 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.187686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:32.189969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 924/1829] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:32.209068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeaac4e-5974-496a-a569-bf58ed85d3c9 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.220183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.223794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.224947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.226152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] PATH_INFO: `/v3/projects/b4e8aaa3c0824d87824e7983e8e747e2/users/0c9750ca456a437aa69df272b632d36e/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.287009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e8aaa3c0824d87824e7983e8e747e2, user_id=0c9750ca456a437aa69df272b632d36e, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.289998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.331498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 906/1830] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:32 2022] PUT /identity/v3/projects/b4e8aaa3c0824d87824e7983e8e747e2/users/0c9750ca456a437aa69df272b632d36e/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:32.343032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf5861-3444-4523-a991-6407236b2829 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.343315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf5861-3444-4523-a991-6407236b2829 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.343692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf5861-3444-4523-a991-6407236b2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.519187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0eaf5861-3444-4523-a991-6407236b2829 None None] MFA Rules not processed for user `0c9750ca456a437aa69df272b632d36e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:32.520419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0eaf5861-3444-4523-a991-6407236b2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:32.703194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 925/1831] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:32.704811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:32.777030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997cd4-fbee-454d-acbe-a325c5e599a8 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.777458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997cd4-fbee-454d-acbe-a325c5e599a8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.778494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07997cd4-fbee-454d-acbe-a325c5e599a8 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.781467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 926/1832] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:27:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:32.809468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.809468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.809468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.881468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:32.882703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:32.892370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50692b-3496-4f86-847a-fd20dcb1fccd admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:32.898303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:32.900782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:32.901204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:32.902212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:32.905149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:32.908868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:32.990967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 907/1833] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:32 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:32.996158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.120227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1b8e7-fad1-4beb-a1f1-44304b1bfcd8 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.126220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.127141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.127543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.128647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.129856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.132495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:33.142829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 927/1834] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:33.167117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 908/1835] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:32 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:33.170859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.198200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876d8d4-a974-4d20-b4bd-f99080d14499 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.235328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.252246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.252717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.253137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.271831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.274087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.294025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:33.324843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:33.344459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 928/1836] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:33.347826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.385818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239c64a-c09c-4933-a102-c823797b585c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.396560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.397904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.398486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.398922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] PATH_INFO: `/v3/projects/2f48da89a153454a8612ce42fa842398/users/7ea442fd15114825be6bdd3670238e49/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.450182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f48da89a153454a8612ce42fa842398, user_id=7ea442fd15114825be6bdd3670238e49, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.459555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:33.517540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 929/1837] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:33 2022] PUT /identity/v3/projects/2f48da89a153454a8612ce42fa842398/users/7ea442fd15114825be6bdd3670238e49/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:33.521202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.527206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5278609-cfcd-4e0b-aeb0-f2636527da12 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.674712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:33.783129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0d824-9d80-456d-a9e1-283cb33088de admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.810555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.810555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.810555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.810555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.811031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.814770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:33.857163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 930/1838] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:33.859152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:33.863015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:33.896744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37b861-085f-4098-8e46-eb8cadc058f0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:33.913101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.914485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.914893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.915290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] PATH_INFO: `/v3/projects/39cf0716d4d644869fb895ba9306c124` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.919403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cf0716d4d644869fb895ba9306c124)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.926509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:33.927142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:33.929518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:33.933254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:33.933711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:33.955794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:33.996916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:34.048354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cf0716d4d644869fb895ba9306c124', 'request_id': 'req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.057391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 909/1839] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 824 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:34.058533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.060318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cf0716d4d644869fb895ba9306c124', 'request_id': 'req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.073509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cf0716d4d644869fb895ba9306c124', 'request_id': 'req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.099910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] The token cache is being invalidate because project 39cf0716d4d644869fb895ba9306c124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:34.099910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.102513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5bfe4636-4a95-44bc-b3d3-29a4c2a199bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.111745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 931/1840] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:33 2022] DELETE /identity/v3/projects/39cf0716d4d644869fb895ba9306c124 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:34.120579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5106-34ee-4458-be40-a0f864132b0b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.123266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d147802-08d4-484a-b22f-77669669d498 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:34.123424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5106-34ee-4458-be40-a0f864132b0b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.125318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e5106-34ee-4458-be40-a0f864132b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.183001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4f3e5106-34ee-4458-be40-a0f864132b0b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:34.185087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4f3e5106-34ee-4458-be40-a0f864132b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.325965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:34.327543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.330111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.331552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] PATH_INFO: `/v3/projects/2f48da89a153454a8612ce42fa842398/users/7ea442fd15114825be6bdd3670238e49/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.338723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f48da89a153454a8612ce42fa842398, user_id=7ea442fd15114825be6bdd3670238e49, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:34.342229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b74169-5b82-4aab-9f42-ea698cef8c99 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:34.370551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 910/1841] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:34 2022] PUT /identity/v3/projects/2f48da89a153454a8612ce42fa842398/users/7ea442fd15114825be6bdd3670238e49/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:34.374584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 932/1842] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:34.380778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b836d9-9a92-4fb6-95f7-3de37c71d7df None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.382611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.383178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b836d9-9a92-4fb6-95f7-3de37c71d7df None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.383669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b836d9-9a92-4fb6-95f7-3de37c71d7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.385672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.386194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.436417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:34.438453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.458229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-54b836d9-9a92-4fb6-95f7-3de37c71d7df None None] MFA Rules not processed for user `7ea442fd15114825be6bdd3670238e49`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:34.459965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-54b836d9-9a92-4fb6-95f7-3de37c71d7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.511042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:34.590858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:34.614989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 933/1843] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:34.614989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.620027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.646830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044bde9-7851-4626-9eb6-2535c277ec5d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:34.657277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:34.658114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.658422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.658691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] PATH_INFO: `/v3/users/4bdd28b9e58742b082d26d4107e00323` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.661571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdd28b9e58742b082d26d4107e00323)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:34.669272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:34.783065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdd28b9e58742b082d26d4107e00323'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.792915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdd28b9e58742b082d26d4107e00323'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.801783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdd28b9e58742b082d26d4107e00323'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.802869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdd28b9e58742b082d26d4107e00323'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.818480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdd28b9e58742b082d26d4107e00323'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:34.826269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 934/1844] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:34 2022] DELETE /identity/v3/users/4bdd28b9e58742b082d26d4107e00323 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:34.827297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.828910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.833640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 911/1845] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:34.835013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.835625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.835625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.855090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:34.856318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.913015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd850-94ba-4614-a808-a20367f0c416 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:34.919919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:34.920714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:34.921141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 912/1846] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:34.921769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.923180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.924601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:34.925087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:34.926402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:34.926744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:34.928478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:34.943434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 935/1847] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:34.944508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:34.947690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:34.995236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb828-bf48-491f-a979-4996cc0edd3a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:34.999858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.000831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.001158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.001444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.004071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.014589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8307e-2615-464a-8a81-81a14637c3ef admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.018742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.019431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.019722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.020002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.030712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.032166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.043145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 936/1848] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:35.043973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.050621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:35.055916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784ff40-8d70-42ec-aac0-ab9644560a0b service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.064619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.065857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.066701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.066989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.067698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.069707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.086390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.089046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.100308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 937/1849] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:35 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:35.102665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 913/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:35.105888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.202534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04c455-84f8-44b7-aa55-2c2c48b966b3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.207943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.208982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.208982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.209353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.210232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.212439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.218527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.227589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d848d4b-8946-4506-a7d0-de57c4551dee service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.233442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:35.240340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 938/1851] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:35 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:35.246176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.261643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462325b-d365-4589-9a65-bc6ba82a8ad6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.266996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.267755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.268119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.268486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.269326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.271265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.286216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 939/1852] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:35.289446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.302566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.303603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.303864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d5cbb-efc7-4128-8a68-2d9a5f3dc9c3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.305381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.305875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.308786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.310189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.311452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.311745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.312029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.312868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.314807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.326351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 940/1853] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:35.330539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.343505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0576-3adf-4b92-a149-a5cdc568c94e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.348529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.349467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.349787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.350029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] PATH_INFO: `/v3/projects/12645c6ec529491ba19d49e1584705a2/users/ea8739b152f54005b18715bd4025f53c/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.370166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12645c6ec529491ba19d49e1584705a2, user_id=ea8739b152f54005b18715bd4025f53c, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.373151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396e7ae-406e-4314-b042-c623ce9309b1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.378299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.382030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.392778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 914/1854] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:35.408662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.410261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.410754 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 941/1855] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:35 2022] PUT /identity/v3/projects/12645c6ec529491ba19d49e1584705a2/users/ea8739b152f54005b18715bd4025f53c/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:35.421466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.421978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.422267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.436362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddab5-4667-4cbf-ab90-6586d52f5935 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.468303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] MFA Rules not processed for user `ea8739b152f54005b18715bd4025f53c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:35.469619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.527444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.528431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.528792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.529198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] PATH_INFO: `/v3/projects/cdb09585bf494540b8f77af8a84a3227` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.530448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb09585bf494540b8f77af8a84a3227)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.532343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.550729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 942/1856] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:35.553707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.574338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4067-96e1-48c7-96f6-79c570ed3f38 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.580758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.581577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.581866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.582147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.582975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.585828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.598202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb09585bf494540b8f77af8a84a3227', 'request_id': 'req-99c5406e-2fe5-4862-93e6-28f8dfd1f067'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.612494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:35.612807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb09585bf494540b8f77af8a84a3227', 'request_id': 'req-99c5406e-2fe5-4862-93e6-28f8dfd1f067'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.630177 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb09585bf494540b8f77af8a84a3227', 'request_id': 'req-99c5406e-2fe5-4862-93e6-28f8dfd1f067'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.635523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:35.639529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 943/1857] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:35 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:35.645806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.670252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] The token cache is being invalidate because project cdb09585bf494540b8f77af8a84a3227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:35.670541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.671616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.674736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 915/1858] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:35 2022] DELETE /identity/v3/projects/cdb09585bf494540b8f77af8a84a3227 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:35.680098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.682558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.761932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1680c78-96f7-420f-b423-c7a4ab4db521 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.769924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.770877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.771338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.771795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.785316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.788037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.828121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 944/1859] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:35 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:35.841774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5406e-2fe5-4862-93e6-28f8dfd1f067 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.844089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.850114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.850210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.851212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.851467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.852635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] PATH_INFO: `/v3/users/ace11b0a48b14b68984e4542b6e1073d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.854152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace11b0a48b14b68984e4542b6e1073d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.856139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.871062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f4d9e-51ab-41c5-a3e5-af203a54395f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.877514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.878422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.878777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.879126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.879945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.881666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.896369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 945/1860] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:35.897921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.914972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5d8dd-56b6-4c6d-98d3-0b2dac05ab9a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:35.921051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:35.922035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:35.922414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:35.922778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] PATH_INFO: `/v3/projects/4f66999f337b460f980441e0c311039c/users/ce99806016de4530b1764d2930816acd/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:35.934323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace11b0a48b14b68984e4542b6e1073d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.942252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f66999f337b460f980441e0c311039c, user_id=ce99806016de4530b1764d2930816acd, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:35.944356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace11b0a48b14b68984e4542b6e1073d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.947040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:35.958960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace11b0a48b14b68984e4542b6e1073d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.960233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace11b0a48b14b68984e4542b6e1073d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.960548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace11b0a48b14b68984e4542b6e1073d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:35.964945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 946/1861] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:35 2022] PUT /identity/v3/projects/4f66999f337b460f980441e0c311039c/users/ce99806016de4530b1764d2930816acd/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:35.968863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:35.971756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:35.989409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 916/1862] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:35 2022] DELETE /identity/v3/users/ace11b0a48b14b68984e4542b6e1073d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:36.045867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0fa8-75ac-476a-a73a-2230fa56236b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.104336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:36.113527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.114214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.114508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.114791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.115542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.118043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cfbc2-e61a-4fa0-a206-9a4c2b17f971 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.128448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 947/1863] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:36.133632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.143634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df75da8-62e6-4f5a-915d-239977ef5b9a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.152264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.153653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.153948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.154234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] PATH_INFO: `/v3/projects/4f66999f337b460f980441e0c311039c/users/ce99806016de4530b1764d2930816acd/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.156753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f66999f337b460f980441e0c311039c, user_id=ce99806016de4530b1764d2930816acd, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.159127 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.170409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 917/1864] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:36 2022] PUT /identity/v3/projects/4f66999f337b460f980441e0c311039c/users/ce99806016de4530b1764d2930816acd/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:36.181427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.181660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.182249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.208467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] MFA Rules not processed for user `ce99806016de4530b1764d2930816acd`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:36.209748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.284389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 948/1865] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:36.291866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.293926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.297381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.298708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.363000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42e6e1-fb5b-4be9-b9a1-17f982c0f0f0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.364387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e9e33-4433-41c0-8cf6-a8e9fdd193d5 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.368484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.368619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.369306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.369418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.369625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.369713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.369909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.370064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] PATH_INFO: `/v3/projects/179d0964c8864891aaa7dc72823db5d2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.371251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=179d0964c8864891aaa7dc72823db5d2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.373188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.374791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:36.379833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.381457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f4015-a03e-472d-bde5-84e9cb9b89ac service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.391163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 949/1866] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:36.428484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179d0964c8864891aaa7dc72823db5d2', 'request_id': 'req-28178505-3c9b-4375-896b-49674212e0f8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.435197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179d0964c8864891aaa7dc72823db5d2', 'request_id': 'req-28178505-3c9b-4375-896b-49674212e0f8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.440770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179d0964c8864891aaa7dc72823db5d2', 'request_id': 'req-28178505-3c9b-4375-896b-49674212e0f8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.456737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] The token cache is being invalidate because project 179d0964c8864891aaa7dc72823db5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:36.457081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.457770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-28178505-3c9b-4375-896b-49674212e0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.459515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 918/1867] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:36 2022] DELETE /identity/v3/projects/179d0964c8864891aaa7dc72823db5d2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:36.464150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.464480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.464725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.488908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:36.490278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.558324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 950/1868] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:36.566143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.566435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.566717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.588748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:36.589994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.598074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 919/1869] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:36.605526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.616436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bdb68-bc1e-40d5-b561-170d2fd391c2 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.622052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.622716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.623003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.623282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.624007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:36.624817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.626221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.643005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 951/1870] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:36.647409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.659808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e88ba-b241-41ab-9a81-145a6e5f7f49 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.664965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.665728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.666081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.666538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.667093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.669251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.698376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 920/1871] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:36.702045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.712305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa6037-25c3-467b-b172-fca6c82b3149 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.716825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:36.784513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.785052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.785346 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.785621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.786356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.788449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.813718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.815974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:36.824687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 952/1872] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:36 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:36.829109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.834102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a40a98-42cf-40d9-b03a-4bb94e1db2ef admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.842349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.844090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.844658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.845142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] PATH_INFO: `/v3/users/ea8739b152f54005b18715bd4025f53c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.847437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8739b152f54005b18715bd4025f53c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.851030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641c85-8476-4a09-ab52-e25dc42203cf admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.851665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.859003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.860079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.860569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.861108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.862550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.871339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.922885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 953/1873] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:36.928692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:36.957655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a7a6e-eb4a-4f80-956a-11bcf907d8fd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:36.973514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8739b152f54005b18715bd4025f53c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:36.980105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:36.980810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:36.981133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:36.981415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:36.982159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:36.984270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:36.989362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8739b152f54005b18715bd4025f53c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:37.031813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8739b152f54005b18715bd4025f53c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:37.036000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8739b152f54005b18715bd4025f53c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:37.036905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-bb9ab437-99b3-4115-92ef-c25443577fcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8739b152f54005b18715bd4025f53c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:37.054397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 954/1874] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:37.063034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:37.067773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:37.111294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 921/1875] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:36 2022] DELETE /identity/v3/users/ea8739b152f54005b18715bd4025f53c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:37.116380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.116696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.117008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.123892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b48497-1614-4499-bae1-29bbb6c3ca46 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:37.197957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:37.207856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:37.278141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:37.332792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 922/1876] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:37.345563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:37.350853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:37.362701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:37.363623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.363916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.364194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] PATH_INFO: `/v3/projects/0e8af3a0248f4eedbd238283f6ccf1d4/users/b924d40e21a441b6901ce4fdf8b8caa4/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.389210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8af3a0248f4eedbd238283f6ccf1d4, user_id=b924d40e21a441b6901ce4fdf8b8caa4, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:37.391997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5e0e1-7f07-4181-8eb9-61777309fd38 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:37.418867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 955/1877] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:37 2022] PUT /identity/v3/projects/0e8af3a0248f4eedbd238283f6ccf1d4/users/b924d40e21a441b6901ce4fdf8b8caa4/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:37.426285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8657fa-9521-4f48-9ce1-51677265d075 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:37.431161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.431518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.431887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.463857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] MFA Rules not processed for user `b924d40e21a441b6901ce4fdf8b8caa4`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:37.465065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:37.495620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:37.496522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.496995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.497371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.552658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:37.600997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 956/1878] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:37.602730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:37.637853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:37.639720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:37.652807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 923/1879] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:37.673494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7c1ab-a719-4bdd-9e59-1b0164cfb9ee None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:37.683628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:37.683628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.684689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.685197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.686690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:37.689684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:37.753772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 957/1880] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:37 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:37.756076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:37.771903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175293ee-a9c6-41b7-b03d-c956e011794b service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:37.828624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:37.866136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:37.867586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.868050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.868475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.869319 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:37.873053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:37.925073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 924/1881] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:37 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:37.929186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:37.942640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e390-f89b-42d4-95f6-5c2129a3918c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:37.949725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:37.950523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:37.950885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:37.951219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:37.952181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:37.953940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:37.984378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 958/1882] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:37.985324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.004089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90cf9c-85a1-4722-a71b-5725849ce033 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.016325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.022797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.023126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.024046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] PATH_INFO: `/v3/projects/a446a66e08ae4b6bb2104134f18afb18/users/4bea550da54c4f48b7253978c99b1db9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.061748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a446a66e08ae4b6bb2104134f18afb18, user_id=4bea550da54c4f48b7253978c99b1db9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.064322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.091282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 925/1883] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:37 2022] PUT /identity/v3/projects/a446a66e08ae4b6bb2104134f18afb18/users/4bea550da54c4f48b7253978c99b1db9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:38.096907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.135413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d99b97-bc2e-4006-9152-e01aae301ba5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.185778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.187683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:38.325035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.326059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.326352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.326891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.327650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.329315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.357938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 959/1884] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:38.363718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.389132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f060d-1a38-4e51-ac96-89ac93e60115 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.400278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.401278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.401645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.401831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] PATH_INFO: `/v3/projects/12645c6ec529491ba19d49e1584705a2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.403960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12645c6ec529491ba19d49e1584705a2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.407021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.407688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6f462-63b7-43ac-95da-2c69adbc615f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.447256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.448178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.448501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.449127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] PATH_INFO: `/v3/projects/a446a66e08ae4b6bb2104134f18afb18/users/4bea550da54c4f48b7253978c99b1db9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.454596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a446a66e08ae4b6bb2104134f18afb18, user_id=4bea550da54c4f48b7253978c99b1db9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.469939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc51636-e610-4890-a483-e85c2677d771 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.501990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 960/1885] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:38 2022] PUT /identity/v3/projects/a446a66e08ae4b6bb2104134f18afb18/users/4bea550da54c4f48b7253978c99b1db9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:38.513679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.513913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.514102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.573104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] MFA Rules not processed for user `4bea550da54c4f48b7253978c99b1db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:38.574347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:38.581796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12645c6ec529491ba19d49e1584705a2', 'request_id': 'req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.605955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12645c6ec529491ba19d49e1584705a2', 'request_id': 'req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.615589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12645c6ec529491ba19d49e1584705a2', 'request_id': 'req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.646178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] The token cache is being invalidate because project 12645c6ec529491ba19d49e1584705a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:38.646694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.647829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.651315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 926/1886] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:38 2022] DELETE /identity/v3/projects/12645c6ec529491ba19d49e1584705a2 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:38.667542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.668565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:38.677339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:38.748514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 961/1887] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:38.763166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:38.767238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:38.772301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:38.831312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058444f0-5afc-4155-82dd-f6fb9ef2ccf4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.854548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.855468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.855748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.856206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] PATH_INFO: `/v3/users/ce99806016de4530b1764d2930816acd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.857414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce99806016de4530b1764d2930816acd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.859264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.871812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b31c3-217b-460e-b601-fc4df32ca8ab None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:38.876645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:38.877476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:38.878334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:38.878713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:38.903034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:38.905380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:38.943877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 962/1888] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:38.948990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce99806016de4530b1764d2930816acd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.962918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce99806016de4530b1764d2930816acd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.980597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce99806016de4530b1764d2930816acd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.981424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce99806016de4530b1764d2930816acd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:38.982121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce99806016de4530b1764d2930816acd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.001379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 927/1889] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:38 2022] DELETE /identity/v3/users/ce99806016de4530b1764d2930816acd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:39.058219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.083196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5dc5-1b72-4ae0-868b-bb47c7995fff service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.236495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.236495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.236495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.236930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.239474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:39.354322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.356765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9a830-8963-44df-bafd-b9901a0eedd6 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.370545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 963/1890] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:39.457488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.460656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:39.547695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b07b46-ac6f-43fb-833f-f56051d1c749 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.561427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.562576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.562932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.563278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] PATH_INFO: `/v3/projects/4f66999f337b460f980441e0c311039c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.564672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f66999f337b460f980441e0c311039c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.566870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.641001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f66999f337b460f980441e0c311039c', 'request_id': 'req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.659128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f66999f337b460f980441e0c311039c', 'request_id': 'req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.661641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:39.669310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f66999f337b460f980441e0c311039c', 'request_id': 'req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.691651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] The token cache is being invalidate because project 4f66999f337b460f980441e0c311039c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:39.691951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.692611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.696983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 928/1891] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:39 2022] DELETE /identity/v3/projects/4f66999f337b460f980441e0c311039c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:39.699710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b71-9c62-4618-a982-616f7b633850 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.700009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b71-9c62-4618-a982-616f7b633850 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.700295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b71-9c62-4618-a982-616f7b633850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.736129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6f672b71-9c62-4618-a982-616f7b633850 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:39.737343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6f672b71-9c62-4618-a982-616f7b633850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:39.754614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.756150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:39.764248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:39.810003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 964/1892] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:39.818583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.818877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.819169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.841059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:39.842601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:39.851113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 965/1893] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:39.855652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.861899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbe7d-d8e3-4428-8fd1-be5d71b77b85 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.865816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a25a-c3b5-4e82-852f-468a5362fd2a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.868288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.869204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.869494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.869847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] PATH_INFO: `/v3/users/b924d40e21a441b6901ce4fdf8b8caa4` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.871062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b924d40e21a441b6901ce4fdf8b8caa4)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.871898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.872604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.872988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.873096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.873276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.874043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:39.874366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.876685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.890787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 966/1894] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:27:39.895040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.905404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c080f6-68ba-4c7d-9af8-699c8469b8f4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:39.910358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:39.910997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.911287 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.911567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.912252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:39.914343 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:39.922035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b924d40e21a441b6901ce4fdf8b8caa4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.930335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b924d40e21a441b6901ce4fdf8b8caa4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.947565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 967/1895] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:39 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:39.948547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:39.949756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b924d40e21a441b6901ce4fdf8b8caa4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.950493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b924d40e21a441b6901ce4fdf8b8caa4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.951137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ffe3391e-c43d-47d6-91a0-0c77b7717554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b924d40e21a441b6901ce4fdf8b8caa4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:39.966435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 929/1896] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:39 2022] DELETE /identity/v3/users/b924d40e21a441b6901ce4fdf8b8caa4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:39.971412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:39.971718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:39.972007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:39.994139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:39.995432 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:40.020123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609ab81-8597-4042-8829-99312e3a32cf admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.065933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 930/1897] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:40.072236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.073640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:40.084808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:40.089588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.090318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.090609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.090893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.091602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.093686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.123037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 968/1898] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:39 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:40.124109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.125797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:40.138036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3151785-051a-4d9f-90b7-76da5d5da7ca admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.143388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2dcb-5be3-4822-9ecb-8a4f110e56c9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.143848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.144556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.144827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.145176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.145923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.147476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.147943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.148417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.148706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.149023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.158431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.162788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.162928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 969/1899] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:40.164742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.170979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 931/1900] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:40.175773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735920e-ad64-4649-bfc9-f52ea73668b9 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.181591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.182266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.182557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.182845 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.183590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.185247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093271c-86f9-42eb-a1fa-b59657889131 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.190840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:40.200896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 970/1901] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:40.202522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.214330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd184b-e300-4434-934c-ad0b41ef7949 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.220197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.221131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.221433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.221988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] PATH_INFO: `/v3/projects/887657b8b7e449faa231960edfcffc15/users/5cf4ee191e3e46c2b8036f6de95588ae/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.240259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=887657b8b7e449faa231960edfcffc15, user_id=5cf4ee191e3e46c2b8036f6de95588ae, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.243733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.260523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 932/1902] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:40 2022] PUT /identity/v3/projects/887657b8b7e449faa231960edfcffc15/users/5cf4ee191e3e46c2b8036f6de95588ae/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:40.271059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.271356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.271643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.301418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] MFA Rules not processed for user `5cf4ee191e3e46c2b8036f6de95588ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:40.302642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:40.403526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 971/1903] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:40.407522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.488401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16669501-e19a-461b-bb9a-de82fc9a4b7f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.495848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.496734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.497123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.497541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.498361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.500622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.537333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 933/1904] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:40 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:40.538131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.560297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e605a9-d7fc-49fe-b195-b76962db0e56 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.646325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.647008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.648132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.648132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.649253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.651476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.682841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 972/1905] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:40 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:40.687016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.700093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c1273-51e0-4b5e-ba32-17cf7f432c01 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.716836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.717665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.717958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.718285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.718989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.720586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.736588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 934/1906] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:40.737874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.751009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e71c-18ad-423b-aceb-d70f9f9f5e1f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.758741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.759807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.760206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.760497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] PATH_INFO: `/v3/projects/f00bda3285a94bc090312ea304625c1e/users/6adac12f823b40b79c56ac5fe8567806/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.783399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00bda3285a94bc090312ea304625c1e, user_id=6adac12f823b40b79c56ac5fe8567806, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.786417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:40.828303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 973/1907] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:40 2022] PUT /identity/v3/projects/f00bda3285a94bc090312ea304625c1e/users/6adac12f823b40b79c56ac5fe8567806/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:40.833257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.849796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf238f9f-4347-466c-a993-962e6fc27b50 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:40.886500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:40.888380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:40.949034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:40.966821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:40.968965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:40.969358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:40.970107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:40.972007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:40.975920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.001786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 935/1908] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:41.006331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:41.016435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:41.019983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e276cfc-4e31-48d7-ab04-2b5d028cbbc1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:41.028191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:41.029768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.030137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.030528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] PATH_INFO: `/v3/projects/f00bda3285a94bc090312ea304625c1e/users/6adac12f823b40b79c56ac5fe8567806/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.032814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00bda3285a94bc090312ea304625c1e, user_id=6adac12f823b40b79c56ac5fe8567806, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:41.035845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f94ef-13e8-4417-9ac1-ee9ba12892bb admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.043744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8bad-2bdc-4f95-bf25-06e793cda9c4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:41.052649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 936/1909] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:41 2022] PUT /identity/v3/projects/f00bda3285a94bc090312ea304625c1e/users/6adac12f823b40b79c56ac5fe8567806/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:41.065564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.065945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.066300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.124813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] MFA Rules not processed for user `6adac12f823b40b79c56ac5fe8567806`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:41.126541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:41.230160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:41.231216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.231975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.232290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] PATH_INFO: `/v3/projects/0e8af3a0248f4eedbd238283f6ccf1d4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.233825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8af3a0248f4eedbd238283f6ccf1d4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:41.236260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.287673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 937/1910] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:41.387033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8af3a0248f4eedbd238283f6ccf1d4', 'request_id': 'req-18b1211a-fd0a-4084-8dfc-6d7b8484e415'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.395105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8af3a0248f4eedbd238283f6ccf1d4', 'request_id': 'req-18b1211a-fd0a-4084-8dfc-6d7b8484e415'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.409196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8af3a0248f4eedbd238283f6ccf1d4', 'request_id': 'req-18b1211a-fd0a-4084-8dfc-6d7b8484e415'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.418997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:41.423331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:41.428116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] The token cache is being invalidate because project 0e8af3a0248f4eedbd238283f6ccf1d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:41.428513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.429357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.431822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 974/1911] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:40 2022] DELETE /identity/v3/projects/0e8af3a0248f4eedbd238283f6ccf1d4 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:41.435827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:41.439500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:41.571776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1211a-fd0a-4084-8dfc-6d7b8484e415 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:41.584780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828d9d-a461-4e2a-be64-a0a39a73598b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:41.588256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:41.590629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.593930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.594938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] PATH_INFO: `/v3/users/4bea550da54c4f48b7253978c99b1db9` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.597760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bea550da54c4f48b7253978c99b1db9)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:41.600128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.685606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:41.696852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:41.697629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:41.697979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:41.698987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:41.701804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:41.706668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bea550da54c4f48b7253978c99b1db9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.722956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bea550da54c4f48b7253978c99b1db9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.737491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bea550da54c4f48b7253978c99b1db9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.739206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bea550da54c4f48b7253978c99b1db9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.767306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bea550da54c4f48b7253978c99b1db9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:41.784493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 975/1912] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:41 2022] DELETE /identity/v3/users/4bea550da54c4f48b7253978c99b1db9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:41.799852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:41.870701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:41.873038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:41.886065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 938/1913] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:42.114239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:42.116358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:42.194310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc4a02-48af-4ecc-a7f5-80b1e85fa206 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:42.199663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:42.200750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:42.201113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:42.201472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:42.203416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:42.343131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:42.344636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8cec5-4e93-419e-b5c1-17a52cbfe8bd service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:42.355738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 976/1914] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:42.838318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:42.858605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101722ef-d8fa-4c28-bca6-d26ed27e273a service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:42.968162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:42.968849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:42.969008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:42.969302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] PATH_INFO: `/v3/projects/a446a66e08ae4b6bb2104134f18afb18` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:42.970772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a446a66e08ae4b6bb2104134f18afb18)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:42.973603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.053904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a446a66e08ae4b6bb2104134f18afb18', 'request_id': 'req-35fa897c-8130-4ffc-9866-216c636998d5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.063111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a446a66e08ae4b6bb2104134f18afb18', 'request_id': 'req-35fa897c-8130-4ffc-9866-216c636998d5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.070173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a446a66e08ae4b6bb2104134f18afb18', 'request_id': 'req-35fa897c-8130-4ffc-9866-216c636998d5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.092220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] The token cache is being invalidate because project a446a66e08ae4b6bb2104134f18afb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:43.092525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.093284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-35fa897c-8130-4ffc-9866-216c636998d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.095233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 939/1915] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:42 2022] DELETE /identity/v3/projects/a446a66e08ae4b6bb2104134f18afb18 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:43.104278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.104278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.104278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.139294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:43.147782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.316348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 977/1916] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:43.326845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.327485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.328054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.357609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:43.359498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.368711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 940/1917] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:43.373202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.394749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f138e8-ac26-4f5a-8af3-16bcc85c5690 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.401505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.402914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.403246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.404474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.405266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:43.405730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.407622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.411067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:43.432571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.443037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f24ed-3bdc-40db-8f28-c11b74a0b3f1 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.444369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 978/1918] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:43.448945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.449719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.450113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.450463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.451237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.453665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.492296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 941/1919] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:43.497184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.528340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32232-08a0-4c89-865f-036327500773 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.674378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.674588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.675245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.675546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.676393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.678445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.686595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.688450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.708197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8da89b-4a1a-46a0-a507-66224e73a12e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.717650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.719001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.719001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.719223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] PATH_INFO: `/v3/users/0c9750ca456a437aa69df272b632d36e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.720786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9750ca456a437aa69df272b632d36e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.725398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.730076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 979/1920] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:43.737280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.762519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb682b-40c8-41a3-b7b0-9cb7f3450a4b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.773365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:43.787251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.794587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.796303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.797070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.797853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.802545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.809533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9750ca456a437aa69df272b632d36e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.824527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 980/1921] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:43.825844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.838929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9750ca456a437aa69df272b632d36e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.857049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9750ca456a437aa69df272b632d36e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.857909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9750ca456a437aa69df272b632d36e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.858761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-03b4cbf1-57ea-4b0c-8d23-024c06730e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9750ca456a437aa69df272b632d36e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:43.859654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635d4e-7352-4785-9d12-100c952c4055 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:43.871242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:43.873543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.874194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.875900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 942/1922] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:43 2022] DELETE /identity/v3/users/0c9750ca456a437aa69df272b632d36e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:43.876134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.877158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:43.881234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:43.881607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:43.881963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:43.882216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:43.915172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 981/1923] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:43.919691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:43.922254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.937433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:43.938923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:43.940853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda83d96-91f0-444a-b570-3f74848defef admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.090716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 943/1924] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:44.090716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.092358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.111263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.114294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.114737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.115030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] PATH_INFO: `/v3/projects/ed681187fb054bab9996998cc6b93a3d/users/298bfffb51da4f8096243cf49cf264ed/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.134479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9258fb3-ae11-4791-b3a2-0aa7f899fac1 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.141530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.142433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.142777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.143062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] PATH_INFO: `/v3/users/5c5a0b753ccd440281d5e64d185e1afd` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.144431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5a0b753ccd440281d5e64d185e1afd)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.146584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.154392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed681187fb054bab9996998cc6b93a3d, user_id=298bfffb51da4f8096243cf49cf264ed, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.162144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.197558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 982/1925] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:43 2022] PUT /identity/v3/projects/ed681187fb054bab9996998cc6b93a3d/users/298bfffb51da4f8096243cf49cf264ed/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:44.199073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.201548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.250133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5a0b753ccd440281d5e64d185e1afd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:44.284338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5a0b753ccd440281d5e64d185e1afd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:44.330824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5a0b753ccd440281d5e64d185e1afd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:44.331358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5a0b753ccd440281d5e64d185e1afd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:44.334730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:44.346357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-978f9bda-bc3c-4ff4-9d48-26762785cda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5a0b753ccd440281d5e64d185e1afd'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:44.352296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d6dc1-07be-4cd3-9e0c-204480f4dbe8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.357897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.359166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.359683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.360161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.363337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.380796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 944/1926] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:44 2022] DELETE /identity/v3/users/5c5a0b753ccd440281d5e64d185e1afd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:44.380796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.380796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.380796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.445362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] MFA Rules not processed for user `298bfffb51da4f8096243cf49cf264ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:44.445362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.460062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:44.531861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.537792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2446a82-8f1b-418d-b794-f1d199f0cda9 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.570511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 983/1927] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:44.574061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.574315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.574600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.615989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:44.617849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.641110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 945/1928] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:44.645436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.646793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.716501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 984/1929] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:44.731469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.735392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.767091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6be89-9afb-454c-861f-1c857daf3f3d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.779847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.781058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.781348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.781631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.782336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.784316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.813249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255c4e-9719-41d0-88d8-ee6dbbb30aac None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.818830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.820403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.820922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 946/1930] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:44 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:44.821451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.822863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.825247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.838047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.839583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.856929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 985/1931] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:44.877520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.878870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:44.897925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5696dc-cceb-4dd6-b2d5-7c5f13ea958f service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.913033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341627c4-6e78-4586-8d90-e1e1aa03f40f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.919411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.922346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.922802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.923133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.924107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.927337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:44.964493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 947/1932] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:44 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:44.965469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:44.982001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344eb76-d9b6-4040-8dcd-66c34fe969e2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:44.990268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:44.991124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:44.991289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:44.991563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:44.992406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:44.995893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.011478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:45.045828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 948/1933] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:45.051863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.071455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.073484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.075016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e68beb-3be0-490b-baf5-5aaafd384727 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.075440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.075838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] PATH_INFO: `/v3/users/5cf4ee191e3e46c2b8036f6de95588ae` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.079540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf4ee191e3e46c2b8036f6de95588ae)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.087734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.091208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.092126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.092522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.092977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] PATH_INFO: `/v3/projects/c46d3f1dac8344ad94368dab634de96f/users/c01fa1abe8824cd7a138fc4bbaa089e6/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.108003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:45.121984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46d3f1dac8344ad94368dab634de96f, user_id=c01fa1abe8824cd7a138fc4bbaa089e6, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.129719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.161045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 949/1934] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:45 2022] PUT /identity/v3/projects/c46d3f1dac8344ad94368dab634de96f/users/c01fa1abe8824cd7a138fc4bbaa089e6/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:45.166631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.214760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf4ee191e3e46c2b8036f6de95588ae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.231479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf4ee191e3e46c2b8036f6de95588ae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.259581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf4ee191e3e46c2b8036f6de95588ae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.260504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf4ee191e3e46c2b8036f6de95588ae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.286013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf4ee191e3e46c2b8036f6de95588ae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.301310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 986/1935] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:44 2022] DELETE /identity/v3/users/5cf4ee191e3e46c2b8036f6de95588ae => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:45.303500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.306374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.347453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beb8b8-9aa0-4718-928f-0e893b2dfe84 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.463913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b97e2-c6af-4819-b39e-c4cd72d1e0df admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.479636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.479636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.479636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.479636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] PATH_INFO: `/v3/projects/c9f7856f3d1d4ced99a13d1bedd14679` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.480718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f7856f3d1d4ced99a13d1bedd14679)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.482907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.509956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.510814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.511204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.511613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.512834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.515123 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74072-211d-4777-b994-b5daa447ae7b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.531079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 950/1936] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:45.532800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268f17-31c6-4c76-932c-ad8e3bd6aee9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.533221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268f17-31c6-4c76-932c-ad8e3bd6aee9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.533647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66268f17-31c6-4c76-932c-ad8e3bd6aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.560975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-66268f17-31c6-4c76-932c-ad8e3bd6aee9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:45.562391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-66268f17-31c6-4c76-932c-ad8e3bd6aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.563817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f7856f3d1d4ced99a13d1bedd14679', 'request_id': 'req-0aa88a4d-05f2-4a62-8711-b73934c36281'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.571142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f7856f3d1d4ced99a13d1bedd14679', 'request_id': 'req-0aa88a4d-05f2-4a62-8711-b73934c36281'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.573690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:45.597572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f7856f3d1d4ced99a13d1bedd14679', 'request_id': 'req-0aa88a4d-05f2-4a62-8711-b73934c36281'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.625244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] The token cache is being invalidate because project c9f7856f3d1d4ced99a13d1bedd14679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:45.627580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.629758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.633925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 987/1937] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:45 2022] DELETE /identity/v3/projects/c9f7856f3d1d4ced99a13d1bedd14679 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:45.637384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.638564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 951/1938] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:45.640235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.640689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.641189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.643219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.643588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 952/1939] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:27:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:27:45.644558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.646583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.665933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa68e2-74b3-40c8-87f7-65e9370af13f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.673838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa88a4d-05f2-4a62-8711-b73934c36281 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.687308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.688492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.688805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.689126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] PATH_INFO: `/v3/projects/c46d3f1dac8344ad94368dab634de96f/users/c01fa1abe8824cd7a138fc4bbaa089e6/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.696922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46d3f1dac8344ad94368dab634de96f, user_id=c01fa1abe8824cd7a138fc4bbaa089e6, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.699097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.699348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a4fda-6eed-4906-b78d-5a57d9cd0b1b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.713850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.714005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.714465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] PATH_INFO: `/v3/users/9731b9935d824810b4d24c0da2e3a900` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.719170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9731b9935d824810b4d24c0da2e3a900)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:45.722420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 988/1940] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:45 2022] PUT /identity/v3/projects/c46d3f1dac8344ad94368dab634de96f/users/c01fa1abe8824cd7a138fc4bbaa089e6/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:45.722848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:45.724114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.724568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.725054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.753871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:45.755411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.789615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:45.823395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9731b9935d824810b4d24c0da2e3a900'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.843118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9731b9935d824810b4d24c0da2e3a900'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.864581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9731b9935d824810b4d24c0da2e3a900'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.866223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9731b9935d824810b4d24c0da2e3a900'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.868470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9731b9935d824810b4d24c0da2e3a900'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:45.886790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 953/1941] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:45 2022] DELETE /identity/v3/users/9731b9935d824810b4d24c0da2e3a900 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:45.887693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.888793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 989/1942] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:27:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:45.891237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.900863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:45.905507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.982481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909849-7ea2-441b-9650-14c8714d02d4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:45.987771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:45.988562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:45.989569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:45.990109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:45.992984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:45.997055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51ac61-6870-4f02-8d71-d0b085420d8d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.111289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.112556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.112927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.113744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] PATH_INFO: `/v3/projects/b4e8aaa3c0824d87824e7983e8e747e2` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.116838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e8aaa3c0824d87824e7983e8e747e2)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.119196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.145360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:46.150689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.154397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f949ca-eab7-4f98-a435-94fd18094063 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.184472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 954/1943] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:46.186168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.186570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.186983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.222047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e8aaa3c0824d87824e7983e8e747e2', 'request_id': 'req-5ac55b92-4cc3-4936-839e-38ad4adec7dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.223513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] MFA Rules not processed for user `c01fa1abe8824cd7a138fc4bbaa089e6`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:46.225095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.238108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e8aaa3c0824d87824e7983e8e747e2', 'request_id': 'req-5ac55b92-4cc3-4936-839e-38ad4adec7dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.250915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e8aaa3c0824d87824e7983e8e747e2', 'request_id': 'req-5ac55b92-4cc3-4936-839e-38ad4adec7dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.272771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] The token cache is being invalidate because project b4e8aaa3c0824d87824e7983e8e747e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:46.273136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.274146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.276659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 990/1944] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:45 2022] DELETE /identity/v3/projects/b4e8aaa3c0824d87824e7983e8e747e2 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:46.277512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.293552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.308280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:46.368931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 955/1945] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:46.368931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.372805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.420830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac55b92-4cc3-4936-839e-38ad4adec7dc admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.436682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.437921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.437921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.438174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.440033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.455670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.457565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.463482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbdb30-8e57-4a4b-b438-00967e798cd5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.469724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.471068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.471668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.471776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 991/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:46.471876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] PATH_INFO: `/v3/users/7ea442fd15114825be6bdd3670238e49` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.472429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.473128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea442fd15114825be6bdd3670238e49)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.473805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.476207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.547825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea442fd15114825be6bdd3670238e49'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.567360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea442fd15114825be6bdd3670238e49'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.592583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea442fd15114825be6bdd3670238e49'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.595076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea442fd15114825be6bdd3670238e49'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.596437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea442fd15114825be6bdd3670238e49'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.619101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a606b-8aa0-4241-9c9e-a9c4a1735459 service nova] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.625110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 956/1947] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:46 2022] DELETE /identity/v3/users/7ea442fd15114825be6bdd3670238e49 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:46.626328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.628065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.630183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.631224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.631224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.631568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.633890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.711434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:46.718810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8c61-816c-4ae9-b2f7-7472527356a0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.733920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:46.734065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.736785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.737257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.738918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] PATH_INFO: `/v3/projects/f6be6498afac4a5383974417020759ff` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.741402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6be6498afac4a5383974417020759ff)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.747361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.771222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.775237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.795718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 992/1948] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:46.815760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:46.815760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:46.834956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29111039-6a59-484e-9c06-0ce7331d1971 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:46.868168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6be6498afac4a5383974417020759ff', 'request_id': 'req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.883294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6be6498afac4a5383974417020759ff', 'request_id': 'req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.922327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6be6498afac4a5383974417020759ff', 'request_id': 'req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.925074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:46.926230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.926930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.927919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] PATH_INFO: `/v3/projects/887657b8b7e449faa231960edfcffc15` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.930615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887657b8b7e449faa231960edfcffc15)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:46.932907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:46.954332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] The token cache is being invalidate because project f6be6498afac4a5383974417020759ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:46.954672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.963542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ff7a0ef0-ef10-411c-b2cf-ed18f5419ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:46.965782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 957/1949] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:46 2022] DELETE /identity/v3/projects/f6be6498afac4a5383974417020759ff => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:46.970113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4b10-3364-4d1a-a8a7-542b55610b18 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:46.970581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4b10-3364-4d1a-a8a7-542b55610b18 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:46.971223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf4b10-3364-4d1a-a8a7-542b55610b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:46.999818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2ddf4b10-3364-4d1a-a8a7-542b55610b18 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.001412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2ddf4b10-3364-4d1a-a8a7-542b55610b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.046717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 958/1950] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:47.050206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.051748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.052124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.066701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887657b8b7e449faa231960edfcffc15', 'request_id': 'req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.074919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887657b8b7e449faa231960edfcffc15', 'request_id': 'req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.087540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887657b8b7e449faa231960edfcffc15', 'request_id': 'req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.099494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.101759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.109515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] The token cache is being invalidate because project 887657b8b7e449faa231960edfcffc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:47.110097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.111279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.113839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 993/1951] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:46 2022] DELETE /identity/v3/projects/887657b8b7e449faa231960edfcffc15 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:47.119584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.121310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.217198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 959/1952] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:47.217961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.235262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b9280-225a-42ee-9650-68f0de2f988d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.247764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.248613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.249005 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.249345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.250370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:47.253379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.255286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.264692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1a3a0-4cfd-4bb2-8d89-e200ce1dcdc3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.271956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.273362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.273830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.274292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] PATH_INFO: `/v3/users/6adac12f823b40b79c56ac5fe8567806` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.275684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6adac12f823b40b79c56ac5fe8567806)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.277963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.298496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 960/1953] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:47.298496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.300113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:47.301354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.322515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98534d20-49bc-44a4-8edd-e09929466577 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.330106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.331038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.336453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.336453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] PATH_INFO: `/v3/projects/2f48da89a153454a8612ce42fa842398` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.336453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f48da89a153454a8612ce42fa842398)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.340925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.385887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6adac12f823b40b79c56ac5fe8567806'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.401037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6adac12f823b40b79c56ac5fe8567806'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.432536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6adac12f823b40b79c56ac5fe8567806'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.435445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6adac12f823b40b79c56ac5fe8567806'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.475424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f48da89a153454a8612ce42fa842398', 'request_id': 'req-42497b5f-c478-4d6d-8776-4eb86354614a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.493501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6adac12f823b40b79c56ac5fe8567806'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.503032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f48da89a153454a8612ce42fa842398', 'request_id': 'req-42497b5f-c478-4d6d-8776-4eb86354614a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.530692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 994/1954] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:27:47 2022] DELETE /identity/v3/users/6adac12f823b40b79c56ac5fe8567806 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:47.534469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.544211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.546810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f48da89a153454a8612ce42fa842398', 'request_id': 'req-42497b5f-c478-4d6d-8776-4eb86354614a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.556472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:47.579059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] The token cache is being invalidate because project 2f48da89a153454a8612ce42fa842398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:47.589345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.590701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-42497b5f-c478-4d6d-8776-4eb86354614a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:47.598565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 961/1955] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:47 2022] DELETE /identity/v3/projects/2f48da89a153454a8612ce42fa842398 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:47.609656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12d669-dee6-4ede-92ff-3f26d96db960 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.609952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12d669-dee6-4ede-92ff-3f26d96db960 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.610522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12d669-dee6-4ede-92ff-3f26d96db960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.681124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9a12d669-dee6-4ede-92ff-3f26d96db960 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.690009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9a12d669-dee6-4ede-92ff-3f26d96db960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.760702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc986a8-22b4-4dd2-b676-7aeac2004c0e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.769829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.770771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.771321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.771780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.773211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.775494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.808519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 962/1956] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:47.816245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.816984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.817286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.826491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 995/1957] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/projects => generated 375 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:47.828637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.835432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.855334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:47.857131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:47.932109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 963/1958] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:27:47.936139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.946491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bbede-66a2-4437-a2ad-d67fea584e1c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.948174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047217-a1af-41c1-8f50-e6587f92ab7c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:47.952687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.953467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:47.953723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.954028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.954221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:47.954312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.954484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:47.954722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:47.955071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.955464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:47.955777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:47.957397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.957644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:47.972914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 964/1959] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:27:47.977297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.988137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 996/1960] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:47.989558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:47.996210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9421fa-bc3e-4d78-82bb-03f52d4cb66c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.001927 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838090b-e833-4e08-8258-b03ee0aadf8b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.003174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.004051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.004441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.004822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.005687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.008506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.010094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.010782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.011072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.011362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.012099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.013753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.015390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.033077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 997/1961] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:27:48.044483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.050272 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.068920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 965/1962] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:47 2022] POST /identity/v3/projects => generated 407 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:27:48.068920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.082327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9149b-172d-4f83-be98-f5390dc7f30f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.139088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.145933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a56a13-b8f9-4e58-bde8-60e07f72ae13 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.154587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.155449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.155719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.156009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.156851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.157156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.158169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.158543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.158733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.158882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] PATH_INFO: `/v3/projects/f00bda3285a94bc090312ea304625c1e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.160911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00bda3285a94bc090312ea304625c1e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.162778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.180779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 966/1963] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:48.180779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.195914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0f4b-c1c3-4e04-8b03-889a3cbdc4c1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.202944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.203630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.203994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.204411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.205224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.207837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.221221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00bda3285a94bc090312ea304625c1e', 'request_id': 'req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:48.232060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00bda3285a94bc090312ea304625c1e', 'request_id': 'req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:48.240738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 967/1964] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/users => generated 304 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.241484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00bda3285a94bc090312ea304625c1e', 'request_id': 'req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:48.241990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.254299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a7685-f692-400d-88f5-db6e1d0c9f77 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.259640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] The token cache is being invalidate because project f00bda3285a94bc090312ea304625c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:27:48.260013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:48.265013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:27:48.270712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 998/1965] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:27:48 2022] DELETE /identity/v3/projects/f00bda3285a94bc090312ea304625c1e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:48.271546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.273866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.323932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.325299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.325588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.326035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] PATH_INFO: `/v3/projects/90a8ae7a658f43a2b5403d89bab2ce0d/users/12b07eca49974f3bb2a765c2892fb68e/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.343534 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a8ae7a658f43a2b5403d89bab2ce0d, user_id=12b07eca49974f3bb2a765c2892fb68e, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.346083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7b073-7eb6-4395-bcac-6cf5d243ebee admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.346371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91898170-a6fe-4728-a5cc-dbe2907bd9cc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.350786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.351522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.351803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.352064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.352807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.354418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.359897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 968/1966] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:48 2022] PUT /identity/v3/projects/90a8ae7a658f43a2b5403d89bab2ce0d/users/12b07eca49974f3bb2a765c2892fb68e/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:48.361678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4508e-2ec1-4ffb-9e02-650aaabab798 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.361678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4508e-2ec1-4ffb-9e02-650aaabab798 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.361907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4508e-2ec1-4ffb-9e02-650aaabab798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.367818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 999/1967] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:48.371539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.379647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a8b4508e-2ec1-4ffb-9e02-650aaabab798 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.380847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a8b4508e-2ec1-4ffb-9e02-650aaabab798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.381333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26887c53-e494-4157-849e-d9cd4d85ae45 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.403214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.443152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.444006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.444408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.444695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.446022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.452640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 969/1968] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:48.452786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604796af-0f85-4363-9f58-d9e596b20fbe admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.452895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e5e0f-4e1e-4ccd-a859-c9fe5cff6abb None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.452895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e5e0f-4e1e-4ccd-a859-c9fe5cff6abb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.452895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e5e0f-4e1e-4ccd-a859-c9fe5cff6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.463930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1000/1969] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:48.465490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.465870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.466219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.479149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c60e5e0f-4e1e-4ccd-a859-c9fe5cff6abb None None] MFA Rules not processed for user `12b07eca49974f3bb2a765c2892fb68e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.480402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c60e5e0f-4e1e-4ccd-a859-c9fe5cff6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.483735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.485049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.493252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1001/1970] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:48.494250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.503316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7b1c7-76a0-4c13-8a8e-b7e9eb25b6ec None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.508434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.509326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.509661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.509884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] PATH_INFO: `/v3/projects/4f39eb151cf54896ac73662b41ba437d/users/23bc0edb84f649ceb22c60b0cb97b859/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.524729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f39eb151cf54896ac73662b41ba437d, user_id=23bc0edb84f649ceb22c60b0cb97b859, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.527391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.530818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.541522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1002/1971] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:48 2022] PUT /identity/v3/projects/4f39eb151cf54896ac73662b41ba437d/users/23bc0edb84f649ceb22c60b0cb97b859/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:48.542340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.550117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 970/1972] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:48.556021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.556595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.556780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.584677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] MFA Rules not processed for user `23bc0edb84f649ceb22c60b0cb97b859`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:48.587262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.620642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01248-6f83-44df-b699-64802e61b89a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.625847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.626619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.626991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.627336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.628581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:27:48.628988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.630696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.644911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1003/1973] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:27:48.645851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.647955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:48.667074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feccab4-0903-49d6-89dc-1c2cb66a339e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.673085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.674158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.674765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.675099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.676077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.678677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.684352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 971/1974] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:48.685974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.699762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96376024-10d6-4b76-a94a-2d0b3a0ee524 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.705889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.706510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.706795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.716364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.716364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.716364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.747428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 972/1975] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.748967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.759428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1004/1976] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.760945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.850010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662628-f011-47fd-ab34-f9a2fbb58643 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.866435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4fad-ae1d-4569-aa51-7578fea870dc admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.871865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.872536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.872821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.873138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.873858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.876010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.880110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.880919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.881313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.881690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.882816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.883805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:48.885047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.910125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1005/1977] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.911419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.927823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7375fc-3f69-4a8d-81b4-827d5bc00cfb admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:48.940571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:48.941728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:48.942244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:48.942377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:48.943104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:48.945775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:48.951755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 973/1978] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/projects => generated 407 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.956140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:48.974263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1006/1979] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/users => generated 295 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:48.977607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.003646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded4b3f-9b75-4bc8-80c3-c2dfc69cd514 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.095508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:49.113668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.114596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.115013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.116518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.118769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.121718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.158207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9fa8-4677-4ac8-8724-e2e9eea3d194 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.162068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1007/1980] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:27:48 2022] POST /identity/v3/users => generated 310 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:27:49.162068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.175800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.176511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.176797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.177340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.177853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.178729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e877fa-282c-43ca-84bf-43cf152b5521 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.180168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.191505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.197339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.197817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.198132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.199062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.201309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.213626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 974/1981] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.213626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.219974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1008/1982] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.220948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.233321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f17ee6-c197-4d98-9ce6-94aba2777a78 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.239022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.239896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.240132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.240425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.241201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.243670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.246388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fe3b-b534-4cff-82f6-a531f5e459c0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.258871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.260042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.260255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.260571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.264108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.268523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.269894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1009/1983] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.282352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.304019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c441e0-e939-4dd1-838b-73f9156176d3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.322779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 975/1984] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.323560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.324478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.324769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.325146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] PATH_INFO: `/v3/projects/c0af887248614e17a9eaa2936a2d36c4/users/56739a7eb67247d1990d35bb315c9658/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.331638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.360872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fc803-6f7b-4301-a352-d13a981b30fb admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.370847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0af887248614e17a9eaa2936a2d36c4, user_id=56739a7eb67247d1990d35bb315c9658, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.374586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.377242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.378168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.378830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.378830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] PATH_INFO: `/v3/projects/1e9a5c7725ce472b9dfcb939ce742fd3/users/9e68823882b94b31943fa0047a2ffeba/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.400741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1010/1985] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:49 2022] PUT /identity/v3/projects/c0af887248614e17a9eaa2936a2d36c4/users/56739a7eb67247d1990d35bb315c9658/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:49.401795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.453881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9a5c7725ce472b9dfcb939ce742fd3, user_id=9e68823882b94b31943fa0047a2ffeba, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.460201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.504818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 976/1986] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:49 2022] PUT /identity/v3/projects/1e9a5c7725ce472b9dfcb939ce742fd3/users/9e68823882b94b31943fa0047a2ffeba/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:49.506748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9e5f-d63d-464e-adb1-e32e7c0adcda admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.509427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.555171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:49.609200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.609732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9387ee-859a-41f7-a4f7-fd1ca698fc36 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.610534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.610883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.611257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] PATH_INFO: `/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c/users/013efe37314d4098a32dc2233860a97f/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.619741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.620454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.620916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.621291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.622572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.623949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417937c3-f4cd-4b34-9d85-edd6cf96be8e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.630870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdd3edd91d14d3e9ffd54c33eb30a2c, user_id=013efe37314d4098a32dc2233860a97f, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.634076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.641807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 977/1987] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.643443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.643744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.644066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.654840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1011/1988] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:49 2022] PUT /identity/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c/users/013efe37314d4098a32dc2233860a97f/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:49.655988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.681613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] MFA Rules not processed for user `9e68823882b94b31943fa0047a2ffeba`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:49.683351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:49.763971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0de71f-3716-40a8-854b-f88d0f69bbac admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.764846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 978/1989] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:49.765791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.772245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.773536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.773958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.774971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] PATH_INFO: `/v3/projects/c0af887248614e17a9eaa2936a2d36c4/users/56739a7eb67247d1990d35bb315c9658/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.776946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0af887248614e17a9eaa2936a2d36c4, user_id=56739a7eb67247d1990d35bb315c9658, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.777815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5e59f-ca82-49f8-af08-a884c615c441 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.779656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.782738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.783441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.783792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.784018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.784760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.786394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.791898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1012/1990] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:49 2022] PUT /identity/v3/projects/c0af887248614e17a9eaa2936a2d36c4/users/56739a7eb67247d1990d35bb315c9658/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:27:49.793007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.794680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:49.800289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 979/1991] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:27:49.801831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:49.812113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50ff17-5987-46f6-9580-0c40d9457212 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.825761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:49.866473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b594c8-09a5-4143-b051-87115d37aae1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:49.870525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.871195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.871483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.871765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.875023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:49.876116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.876409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.876694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] PATH_INFO: `/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c/users/013efe37314d4098a32dc2233860a97f/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.878982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdd3edd91d14d3e9ffd54c33eb30a2c, user_id=013efe37314d4098a32dc2233860a97f, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.881777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6ddba-a272-48ed-ab82-1ac8488c8b82 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.904963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 980/1992] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:27:49 2022] PUT /identity/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c/users/013efe37314d4098a32dc2233860a97f/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:27:49.906919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.907220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.907510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.942847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:49.949664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] MFA Rules not processed for user `56739a7eb67247d1990d35bb315c9658`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:49.950131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:49.951032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:49.951973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25c530-1ba4-48b0-9e69-f5a6ea5c9df2 service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:49.962606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1013/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:49.964004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:49.964545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:49.964654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:49.994963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] MFA Rules not processed for user `013efe37314d4098a32dc2233860a97f`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:27:49.996307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:50.039105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 981/1994] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:50.049599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:50.050976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:50.084782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1014/1995] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:27:50.115227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34370655-c092-4ad7-b21b-cc97ccbaf457 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:50.119666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:50.120335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:50.120622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:50.121932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:50.183405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:27:50.196976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:50.198663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:50.209633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 982/1996] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:27:50.482471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:50.484246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:50.512200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c663-f4e4-44c5-9d99-a0bee973c153 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:50.522675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:50.523624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:50.523921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:50.524231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:50.544567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:50.549374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:50.582377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1015/1997] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:50 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:51.269048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:51.278412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3bb56-d1e9-4a81-9164-7bf67363d2cf service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:51.282399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:51.283475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:51.283845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:51.284225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:51.372652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:51.377673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:51.402191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 983/1998] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:51 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:52.120747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:52.129679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6302d-d530-43af-89e0-998dbbfaa8f6 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:52.134763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:52.135438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:52.135731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:52.136014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:52.139141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:27:52.229824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:52.231460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:52.249117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1016/1999] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:27:55.446431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:27:55.458837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61257e86-db41-41aa-892d-8991d553ba2a service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:27:55.464841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:27:55.465924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:27:55.466274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:27:55.466650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:27:55.480463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:27:55.482317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e74abd-d300-4746-9d14-2d40703a1704 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:27:55.501016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 984/2000] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:27:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:02.239065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=73 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:05.489189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=38 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:12.766237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:12.858011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995812a-c2d4-4e92-ae66-c2dc0175b0d0 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:12.862765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:12.863752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:12.863944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:12.864215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] PATH_INFO: `/v3/users/23bc0edb84f649ceb22c60b0cb97b859` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:12.865630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bc0edb84f649ceb22c60b0cb97b859)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:12.869141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:12.961050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bc0edb84f649ceb22c60b0cb97b859'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:12.974481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bc0edb84f649ceb22c60b0cb97b859'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:12.983869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bc0edb84f649ceb22c60b0cb97b859'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:12.984708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bc0edb84f649ceb22c60b0cb97b859'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:12.999314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bc0edb84f649ceb22c60b0cb97b859'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.006071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1017/2001] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:12 2022] DELETE /identity/v3/users/23bc0edb84f649ceb22c60b0cb97b859 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:13.010632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:13.011398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:13.011398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:13.030822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:13.032132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:13.169939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 985/2002] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:13.181065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:13.182834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:13.241844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4305e60-0049-425e-b760-94e7192ebb4d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:13.246520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:13.247184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:13.247476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:13.247799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:13.262374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:13.263834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:13.275507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1018/2003] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:13.571972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:13.573531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:13.585900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218c2e7-002e-4543-8656-bdb003ae78bf None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:13.592221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:13.596030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:13.596030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:13.596030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] PATH_INFO: `/v3/projects/4f39eb151cf54896ac73662b41ba437d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:13.596030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f39eb151cf54896ac73662b41ba437d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:13.596826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:13.659678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f39eb151cf54896ac73662b41ba437d', 'request_id': 'req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.667310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f39eb151cf54896ac73662b41ba437d', 'request_id': 'req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.673574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f39eb151cf54896ac73662b41ba437d', 'request_id': 'req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.687220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] The token cache is being invalidate because project 4f39eb151cf54896ac73662b41ba437d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:13.687642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.688306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.690003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 986/2004] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:13 2022] DELETE /identity/v3/projects/4f39eb151cf54896ac73662b41ba437d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:13.693936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:13.704695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90974aaf-cece-4e39-9b8e-ae377cb34dbe service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:13.723627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:13.764701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:13.765611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:13.765903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:13.766185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] PATH_INFO: `/v3/users/013efe37314d4098a32dc2233860a97f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:13.767375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013efe37314d4098a32dc2233860a97f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:13.769336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:13.817641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013efe37314d4098a32dc2233860a97f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.826063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013efe37314d4098a32dc2233860a97f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.833169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013efe37314d4098a32dc2233860a97f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.833913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013efe37314d4098a32dc2233860a97f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.850902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b55f2e1-ea9c-491e-ac12-7a53ce038bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013efe37314d4098a32dc2233860a97f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:13.857083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1019/2005] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:13 2022] DELETE /identity/v3/users/013efe37314d4098a32dc2233860a97f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:14.283921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.286236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:14.308915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3689c-c05d-4dbc-8703-1f22d0e8f1d9 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.407102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.407975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.408326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.408533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] PATH_INFO: `/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.409778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdd3edd91d14d3e9ffd54c33eb30a2c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.411962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.446983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:14.464807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdd3edd91d14d3e9ffd54c33eb30a2c', 'request_id': 'req-d9f48245-b140-4efa-b26a-94ce8b358a69'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:14.470901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdd3edd91d14d3e9ffd54c33eb30a2c', 'request_id': 'req-d9f48245-b140-4efa-b26a-94ce8b358a69'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:14.476306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdd3edd91d14d3e9ffd54c33eb30a2c', 'request_id': 'req-d9f48245-b140-4efa-b26a-94ce8b358a69'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:14.490081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] The token cache is being invalidate because project fcdd3edd91d14d3e9ffd54c33eb30a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:14.490378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:14.491012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d9f48245-b140-4efa-b26a-94ce8b358a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:14.493866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 987/2006] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:14 2022] DELETE /identity/v3/projects/fcdd3edd91d14d3e9ffd54c33eb30a2c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:14.498813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.499372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.499372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.518109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:14.519574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:14.587083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1020/2007] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:14.595449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.595745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.596068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.614934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:14.616433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:14.625966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 988/2008] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:14.630588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.641460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73b1bf-9f1c-4001-9b99-afd5b423a18c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.646727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.647722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.648011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.648296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.649402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:14.649712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.651618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.663767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1021/2009] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:14.669609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.678856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb1bab-0a26-4a2e-a6a7-3d6fad9fc042 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.684512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.685221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.685509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.685789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.686503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.688433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.715572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 989/2010] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:14 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:14.719584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.730240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cb61-1987-45cf-b383-2f9b6e5f17a3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.755606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:14.786004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.786673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.786959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.787238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.787934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.790585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.809838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1022/2011] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:14 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:14.813958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.824718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93136d30-7d95-4b0e-8684-66a2c3ab5926 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.829669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.830365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.830671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.830949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.831680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.833316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.844277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 990/2012] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:14.848259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.858577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86328fab-9f5b-49f6-8379-f9c96fa0f78c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.863094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.863755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.864041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.864321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.865142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.866738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf660d4-c223-4de4-9dcc-33ddd7f7614d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.877292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1023/2013] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:14.881409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:14.891659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb042da-6130-4876-8523-51a46f6354c1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:14.896187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:14.897509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.897793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.898067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] PATH_INFO: `/v3/projects/32f1a3658f4944a2b963e13a9d3d1438/users/7dd8e31ee53541b1ba1c368f36d12e2f/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.912379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f1a3658f4944a2b963e13a9d3d1438, user_id=7dd8e31ee53541b1ba1c368f36d12e2f, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:14.915082 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:14.927639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 991/2014] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:14 2022] PUT /identity/v3/projects/32f1a3658f4944a2b963e13a9d3d1438/users/7dd8e31ee53541b1ba1c368f36d12e2f/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:14.938407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:14.938705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:14.938989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:14.965341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] MFA Rules not processed for user `7dd8e31ee53541b1ba1c368f36d12e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:14.966582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.028423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1024/2015] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:15.033081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.093373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0732e-09d2-4993-8804-55ebec8564cc admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.097944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.098597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.098881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.099159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.099860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.101884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.128719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 992/2016] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:15 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:15.130629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.141606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed06bbb-2002-4ca6-9b74-1fbf81e28e0f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.195201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.196692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.196998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.197281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.197973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.199884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.222627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1025/2017] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:15 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:15.223746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.234918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aab79d-6ec5-44cd-9950-fd0cf0a5bb69 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.239543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.240208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.240496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.240775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.241537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.243936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.255644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 993/2018] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:15.259847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.269887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929e9e0-d6f4-4c19-a262-2d8b568d705b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.274979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.275860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.276147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.276543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] PATH_INFO: `/v3/projects/841b46dcf27d4250b9649e4b0e173c25/users/2ad843fe0d3449949a88037452301415/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.292435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=841b46dcf27d4250b9649e4b0e173c25, user_id=2ad843fe0d3449949a88037452301415, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.295090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.310447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1026/2019] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:15 2022] PUT /identity/v3/projects/841b46dcf27d4250b9649e4b0e173c25/users/2ad843fe0d3449949a88037452301415/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:15.314332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.324355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2069f7-1d67-4434-a189-ef81316e996a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.344709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:15.390569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.391256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.391544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.391828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.392561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.394230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.411311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 994/2020] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:15.415905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.423896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:15.429826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c382ce-aaca-4210-a6f8-8deca29ad50c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.437355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.438508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.438986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.439509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] PATH_INFO: `/v3/projects/841b46dcf27d4250b9649e4b0e173c25/users/2ad843fe0d3449949a88037452301415/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.455452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=841b46dcf27d4250b9649e4b0e173c25, user_id=2ad843fe0d3449949a88037452301415, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.458186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d307db1-792f-42b1-9f10-cbb45cc28b2a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.470713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.471136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1027/2021] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:15 2022] PUT /identity/v3/projects/841b46dcf27d4250b9649e4b0e173c25/users/2ad843fe0d3449949a88037452301415/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:15.472570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.482533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.482763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.483043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.485915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452390-0525-4503-83f4-b832279de7cd admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.509641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] MFA Rules not processed for user `2ad843fe0d3449949a88037452301415`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:15.510861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.542018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.542903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.543186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.543463 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] PATH_INFO: `/v3/users/298bfffb51da4f8096243cf49cf264ed` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.544653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] RBAC: Authorizing `identity:delete_user(user_id=298bfffb51da4f8096243cf49cf264ed)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.546592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.584813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1028/2022] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:15.596584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.598107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.618345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298bfffb51da4f8096243cf49cf264ed'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:15.626626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298bfffb51da4f8096243cf49cf264ed'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:15.643023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298bfffb51da4f8096243cf49cf264ed'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:15.643896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298bfffb51da4f8096243cf49cf264ed'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:15.644512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6a237774-722c-4cf6-ba2e-34c9dc6ff26a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298bfffb51da4f8096243cf49cf264ed'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:15.657531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 995/2023] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:15 2022] DELETE /identity/v3/users/298bfffb51da4f8096243cf49cf264ed => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:15.661808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.662102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.662384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.664898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341dbdd-593d-4f33-a23f-330726be14df None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.679496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:15.681014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.712308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.713002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.713286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.713561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.715388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.716218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:15.746051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 996/2024] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:15.755623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:15.756980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:15.766042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:15.786519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.788087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.798351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1029/2025] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:15.818891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04c864-ab8f-4143-81df-6c058161e889 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:15.822894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:15.823555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:15.823845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:15.824130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:15.838072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:15.839680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:15.849640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 997/2026] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:16.121354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.122744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.138444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b924934-7b3d-44e0-ba59-7799f12e9bdc service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.143679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.144567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.144856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.145178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] PATH_INFO: `/v3/projects/ed681187fb054bab9996998cc6b93a3d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.147085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed681187fb054bab9996998cc6b93a3d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.149389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.199037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed681187fb054bab9996998cc6b93a3d', 'request_id': 'req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.205794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed681187fb054bab9996998cc6b93a3d', 'request_id': 'req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.215216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed681187fb054bab9996998cc6b93a3d', 'request_id': 'req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.224978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.227067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.230399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] The token cache is being invalidate because project ed681187fb054bab9996998cc6b93a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:16.230693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.231334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.232847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1030/2027] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:16 2022] DELETE /identity/v3/projects/ed681187fb054bab9996998cc6b93a3d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:16.236458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.237739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.240186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808150d2-cb5a-48de-b270-52e4a98c7c87 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.308070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6aaf6-17a6-4cf2-a76c-0cf638f7075c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.309111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.310008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.310296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.310578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] PATH_INFO: `/v3/users/7dd8e31ee53541b1ba1c368f36d12e2f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.311786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd8e31ee53541b1ba1c368f36d12e2f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.312947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.313752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.313861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.314098 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.314382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] PATH_INFO: `/v3/users/c01fa1abe8824cd7a138fc4bbaa089e6` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.315584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01fa1abe8824cd7a138fc4bbaa089e6)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.317536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.365013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd8e31ee53541b1ba1c368f36d12e2f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.366421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01fa1abe8824cd7a138fc4bbaa089e6'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.374182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd8e31ee53541b1ba1c368f36d12e2f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.379947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01fa1abe8824cd7a138fc4bbaa089e6'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.388770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01fa1abe8824cd7a138fc4bbaa089e6'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.389114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd8e31ee53541b1ba1c368f36d12e2f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.389703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01fa1abe8824cd7a138fc4bbaa089e6'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.389889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd8e31ee53541b1ba1c368f36d12e2f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.390557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-913f9cf1-4189-44e6-9880-d363d831f178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd8e31ee53541b1ba1c368f36d12e2f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.392273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:16.407068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01fa1abe8824cd7a138fc4bbaa089e6'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.407271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 998/2028] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:16 2022] DELETE /identity/v3/users/7dd8e31ee53541b1ba1c368f36d12e2f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:16.412239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.412239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.412466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.414764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1031/2029] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:16 2022] DELETE /identity/v3/users/c01fa1abe8824cd7a138fc4bbaa089e6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:16.419884 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:16.435069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:16.436332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.556075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 999/2030] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:16.574482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.576361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.642839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8273a8-7508-4447-8710-98ccdbcd9986 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.647654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.649139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.649658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.650418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.662054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.663501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e750a315-3fcb-40d7-bb10-799ac3c8cdf9 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.679841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1032/2031] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:16.756497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.758504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.783281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc74e27-fd65-4009-9ca8-d8fbb82909e8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.789437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.790510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.790800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.791079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] PATH_INFO: `/v3/projects/c46d3f1dac8344ad94368dab634de96f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.792302 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46d3f1dac8344ad94368dab634de96f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:16.794736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:16.853345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46d3f1dac8344ad94368dab634de96f', 'request_id': 'req-663568e3-59db-46d4-8fa1-077dcc60d264'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.859911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46d3f1dac8344ad94368dab634de96f', 'request_id': 'req-663568e3-59db-46d4-8fa1-077dcc60d264'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.865456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46d3f1dac8344ad94368dab634de96f', 'request_id': 'req-663568e3-59db-46d4-8fa1-077dcc60d264'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.879652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] The token cache is being invalidate because project c46d3f1dac8344ad94368dab634de96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:16.879948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.880588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:16.882181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1000/2032] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:16 2022] DELETE /identity/v3/projects/c46d3f1dac8344ad94368dab634de96f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:16.888916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302f88c-4272-4b14-a11c-6b384222fd9c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.889999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302f88c-4272-4b14-a11c-6b384222fd9c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.890325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302f88c-4272-4b14-a11c-6b384222fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.910215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d302f88c-4272-4b14-a11c-6b384222fd9c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:16.911432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d302f88c-4272-4b14-a11c-6b384222fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.970519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:16.972482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:16.973184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1033/2033] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:16.981875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.982179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.982468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.988857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663568e3-59db-46d4-8fa1-077dcc60d264 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:16.995437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:16.996386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:16.996671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:16.996972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] PATH_INFO: `/v3/projects/32f1a3658f4944a2b963e13a9d3d1438` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:16.998900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f1a3658f4944a2b963e13a9d3d1438)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.001040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.002294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:17.003661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.012527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1034/2034] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:17.017738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.027909 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bac82-689f-4fd6-a113-a25613962e16 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.032892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.033577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.033868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.034196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.035575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:17.035894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.037552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.052559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1035/2035] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:28:17.053490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.055655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f1a3658f4944a2b963e13a9d3d1438', 'request_id': 'req-d74ee635-503a-447c-b54c-176f8aadbba2'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.062391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f1a3658f4944a2b963e13a9d3d1438', 'request_id': 'req-d74ee635-503a-447c-b54c-176f8aadbba2'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.063905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe2223-ce5a-437e-a453-2410c90d593c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.068096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f1a3658f4944a2b963e13a9d3d1438', 'request_id': 'req-d74ee635-503a-447c-b54c-176f8aadbba2'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.081068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] The token cache is being invalidate because project 32f1a3658f4944a2b963e13a9d3d1438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:17.081297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.081957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.083519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1001/2036] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:16 2022] DELETE /identity/v3/projects/32f1a3658f4944a2b963e13a9d3d1438 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:17.088114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.089447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.100113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:17.123141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.123804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.124090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.124368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.125092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.127062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.137741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:17.158203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1036/2037] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:17.159266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.160904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.162439 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee635-503a-447c-b54c-176f8aadbba2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.219358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.220231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.220513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.220785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] PATH_INFO: `/v3/users/2ad843fe0d3449949a88037452301415` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.222071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad843fe0d3449949a88037452301415)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.223474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa80674-aecf-4b44-8c79-67d50f988eb1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.224003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.228291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.228993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.229293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.229574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.230267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.232184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.253584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1037/2038] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:17.258604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.269006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5681d-6c95-42ac-8030-0d062e74e0aa admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.274896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.275057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad843fe0d3449949a88037452301415'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.275559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.275854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.276162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.276917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.278528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.282615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad843fe0d3449949a88037452301415'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.293166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1038/2039] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:17.294199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.299776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad843fe0d3449949a88037452301415'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.300495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad843fe0d3449949a88037452301415'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.301128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad843fe0d3449949a88037452301415'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.304063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8ac61-85d3-4477-b3d0-cce5cca0495d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.309038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.309681 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.309970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.310249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.310999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.312615 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969da936-1887-469e-8153-5d888c753f45 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.313572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1002/2040] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:17 2022] DELETE /identity/v3/users/2ad843fe0d3449949a88037452301415 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:17.324483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1039/2041] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:17.328773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.330258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.372742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:17.401164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913929-b5ff-4a01-a0fe-bc0c5b8116ff admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.406031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.406938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.407228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.407531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] PATH_INFO: `/v3/projects/fab1b0b75d3f47ac9e3757ee1e0e1d05/users/fa42c494071b4e328c13d15bea0a6ff7/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.423216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab1b0b75d3f47ac9e3757ee1e0e1d05, user_id=fa42c494071b4e328c13d15bea0a6ff7, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.425933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.439364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1003/2042] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:17 2022] PUT /identity/v3/projects/fab1b0b75d3f47ac9e3757ee1e0e1d05/users/fa42c494071b4e328c13d15bea0a6ff7/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:17.451198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.451492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.451773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.483853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] MFA Rules not processed for user `fa42c494071b4e328c13d15bea0a6ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:17.485309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.553451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1040/2043] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:17.567083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.571493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.630833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:17.632317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.662827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56b0de-6115-46c1-86c5-1472fed4bfd3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.669196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.670845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.671259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.671620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.682481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.684484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120e983-f683-4557-a8b1-52e174de0fa3 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.699785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1004/2044] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:17.754484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:17.777260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7470e-1e31-427d-9fcd-fcff55c4d031 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:17.783374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:17.784339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.784694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.785164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] PATH_INFO: `/v3/projects/841b46dcf27d4250b9649e4b0e173c25` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.786410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=841b46dcf27d4250b9649e4b0e173c25)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:17.788305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:17.845583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841b46dcf27d4250b9649e4b0e173c25', 'request_id': 'req-f13a2d91-1fee-4720-a618-59dec346ccdc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.853273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841b46dcf27d4250b9649e4b0e173c25', 'request_id': 'req-f13a2d91-1fee-4720-a618-59dec346ccdc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.862672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841b46dcf27d4250b9649e4b0e173c25', 'request_id': 'req-f13a2d91-1fee-4720-a618-59dec346ccdc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.874751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] The token cache is being invalidate because project 841b46dcf27d4250b9649e4b0e173c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:17.875000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.875646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f13a2d91-1fee-4720-a618-59dec346ccdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:17.877458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1041/2045] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:17 2022] DELETE /identity/v3/projects/841b46dcf27d4250b9649e4b0e173c25 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:17.882122 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.882482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.883331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:17.903937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:17.905430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:17.974333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1005/2046] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:17.982970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45042-1d6a-410e-a244-447593a3706e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:17.983453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45042-1d6a-410e-a244-447593a3706e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:17.983775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45042-1d6a-410e-a244-447593a3706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.005160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-33f45042-1d6a-410e-a244-447593a3706e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:18.007076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-33f45042-1d6a-410e-a244-447593a3706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:18.016285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1042/2047] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:18.021092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.036739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fe8f2-4527-4553-a405-350e080e6a71 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.042067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.042740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.043044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.043329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.044074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:18.044545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.046124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.064925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1006/2048] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:18.066193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45042-1d6a-410e-a244-447593a3706e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.076740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45042-1d6a-410e-a244-447593a3706e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.081717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.082426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.082671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.082964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.083625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.085607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.113727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1043/2049] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:18 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:18.119011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.129013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0774d-70bf-457d-a8af-2d4a4d346a7e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.174812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:18.188666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.189389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.189663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.189970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.190660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.192577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.215517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1007/2050] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:18 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:18.219601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.230365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9bb3a-627a-436f-8c6a-393efeec99d4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.235295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.235954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.236239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.236516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.237276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.238893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.250153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1044/2051] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:18.254134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.264203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b92604-e866-4185-9569-0edcf7f07e6a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.268675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.269366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.269654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.269936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.270662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.272500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaa98b-0aad-4cd8-b1d5-7d0b7526f1d0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.282322 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1008/2052] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:18.286209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.296560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c640d-3824-4503-89cd-dbcb41b5988a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.301069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.302015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.302368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.302673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] PATH_INFO: `/v3/projects/5313813c5d2e4f01915e812d4665ad02/users/073e63a99e224eb58626484f889693ea/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.317572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5313813c5d2e4f01915e812d4665ad02, user_id=073e63a99e224eb58626484f889693ea, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.320457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.334106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1045/2053] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:18 2022] PUT /identity/v3/projects/5313813c5d2e4f01915e812d4665ad02/users/073e63a99e224eb58626484f889693ea/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:18.346077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.346367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.346678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.375405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] MFA Rules not processed for user `073e63a99e224eb58626484f889693ea`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:18.376623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:18.447207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1009/2054] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:18.458300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:18.460026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:18.518669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6266dbc-7bb4-4f93-a16f-67de226f8168 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:18.522916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:18.523688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:18.524014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:18.524323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:18.533566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:18.535020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e8973-076e-493f-8812-624eec928ff8 service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:18.544946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1046/2055] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:28.443116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:28.540675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:37.486494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:37.488022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:37.563238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e29a9-5fe2-4393-93ef-4b2933821397 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:37.569715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:37.571165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:37.571165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:37.571512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] PATH_INFO: `/v3/users/12b07eca49974f3bb2a765c2892fb68e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:37.572799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b07eca49974f3bb2a765c2892fb68e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:37.575054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:37.627366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b07eca49974f3bb2a765c2892fb68e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:37.639194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b07eca49974f3bb2a765c2892fb68e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:37.655815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b07eca49974f3bb2a765c2892fb68e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:37.656870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b07eca49974f3bb2a765c2892fb68e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:37.657414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b07eca49974f3bb2a765c2892fb68e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:37.674676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1010/2056] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:37 2022] DELETE /identity/v3/users/12b07eca49974f3bb2a765c2892fb68e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:37.680500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:37.681088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:37.681088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:37.703048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:37.707987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:37.782916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1047/2057] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:37.793156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:37.794578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:37.857457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165ab9e-01d0-4009-9681-a06822fed8e3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:37.861399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:37.862059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:37.862349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:37.862630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:37.872608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:37.874107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:37.884343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1011/2058] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:38.194013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:38.196016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:38.214401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d96732-f444-44a2-a168-af2ad7753742 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:38.220285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:38.222223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:38.222940 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:38.223315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] PATH_INFO: `/v3/projects/90a8ae7a658f43a2b5403d89bab2ce0d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:38.230041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a8ae7a658f43a2b5403d89bab2ce0d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:38.233527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:38.288260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a8ae7a658f43a2b5403d89bab2ce0d', 'request_id': 'req-a0bdee49-7091-4a88-8ce2-da32d451c821'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.294023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a8ae7a658f43a2b5403d89bab2ce0d', 'request_id': 'req-a0bdee49-7091-4a88-8ce2-da32d451c821'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.304892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a8ae7a658f43a2b5403d89bab2ce0d', 'request_id': 'req-a0bdee49-7091-4a88-8ce2-da32d451c821'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.318018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] The token cache is being invalidate because project 90a8ae7a658f43a2b5403d89bab2ce0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:38.318440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.319120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.321380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1048/2059] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:38 2022] DELETE /identity/v3/projects/90a8ae7a658f43a2b5403d89bab2ce0d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:38.325562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:38.335443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672db416-7095-49e6-91f1-6f6a6d65715d service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:38.352993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:38.394378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:38.395054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:38.395347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:38.395626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] PATH_INFO: `/v3/users/56739a7eb67247d1990d35bb315c9658` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:38.396827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56739a7eb67247d1990d35bb315c9658)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:38.399166 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:38.467796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56739a7eb67247d1990d35bb315c9658'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.479733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56739a7eb67247d1990d35bb315c9658'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.498086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56739a7eb67247d1990d35bb315c9658'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.498807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56739a7eb67247d1990d35bb315c9658'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.499415 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c32e2379-6136-4141-a17b-8af345c500c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56739a7eb67247d1990d35bb315c9658'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:38.515377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1012/2060] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:38 2022] DELETE /identity/v3/users/56739a7eb67247d1990d35bb315c9658 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:39.049451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.051849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:39.075854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdee49-7091-4a88-8ce2-da32d451c821 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.149282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.151096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.151393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.151675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] PATH_INFO: `/v3/projects/c0af887248614e17a9eaa2936a2d36c4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.153278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0af887248614e17a9eaa2936a2d36c4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.155579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.189561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:39.209190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0af887248614e17a9eaa2936a2d36c4', 'request_id': 'req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:39.215120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0af887248614e17a9eaa2936a2d36c4', 'request_id': 'req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:39.223286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0af887248614e17a9eaa2936a2d36c4', 'request_id': 'req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:39.243752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] The token cache is being invalidate because project c0af887248614e17a9eaa2936a2d36c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:39.244195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:39.255379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7ccb8595-a0d6-4e74-a7b6-cecfa87b0624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:39.255379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1049/2061] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:39 2022] DELETE /identity/v3/projects/c0af887248614e17a9eaa2936a2d36c4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:39.258482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.258783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.259069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.277206 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:39.278501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:39.362730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1013/2062] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:39.371263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.371633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.371842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.400244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:39.401915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:39.415821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1050/2063] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:39.417264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.432157 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca830f0-5f4a-4099-a1eb-cbcf601b3e9e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.446207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.447421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.447700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.448169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.449693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:39.449940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.455652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.475082 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1014/2064] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:39.481309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.495653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc1cfb-bf80-4a00-be5c-c3b6b25eaf8b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.504978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.505739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.507289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.508612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.511302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.516630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.555481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1051/2065] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:39 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:39.557114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.572738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef5b0d-1ec3-43b5-8cc0-7d28cf234350 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.642732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:39.693755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.694999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.695293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.695665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.696536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.698805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.726551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1015/2066] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:39 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:39.731358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.742752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aa745-9acd-49da-8a7b-857b995182f5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.748568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.749581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.749861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.750160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.750939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.752631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.769561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1052/2067] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:39.774944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.789284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41285b07-010b-4a81-ae4c-550fc8a98f66 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.795287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.796048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.796399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.796769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.797808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.799485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.811500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1016/2068] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:39.815767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.826424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54ddc4-bcb6-4743-9fb0-78ecf71b0499 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.831784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.832667 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.832988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.833264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] PATH_INFO: `/v3/projects/00f8e22a430c4791b1e40fe46ebab457/users/791bed922944437280aaa84ab5eef3db/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.855314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f8e22a430c4791b1e40fe46ebab457, user_id=791bed922944437280aaa84ab5eef3db, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.858215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b0eea-61f0-4f3d-ab61-f5484d811e0d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:39.867843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:39.870182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:39.874381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1053/2069] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:39 2022] PUT /identity/v3/projects/00f8e22a430c4791b1e40fe46ebab457/users/791bed922944437280aaa84ab5eef3db/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:39.885177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.885473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.885907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.889294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d2a1d-a076-4c17-a725-9b0ce1400ac5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:39.924804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] MFA Rules not processed for user `791bed922944437280aaa84ab5eef3db`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:39.926823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:39.966556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:39.968020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:39.968500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:39.969080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] PATH_INFO: `/v3/users/0dd5979c1b2a4b75bf0e5afb5faf2026` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:39.972084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd5979c1b2a4b75bf0e5afb5faf2026)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:39.974652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.026965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1054/2070] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:40.032067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.046078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd5979c1b2a4b75bf0e5afb5faf2026'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:40.049959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168be71-9ffe-4063-aa4c-f731cece172d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.056736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.057445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.058233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.058233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.058838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.059666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd5979c1b2a4b75bf0e5afb5faf2026'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:40.062731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.083939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd5979c1b2a4b75bf0e5afb5faf2026'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:40.086243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd5979c1b2a4b75bf0e5afb5faf2026'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:40.087257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eae27704-c1a3-4b6a-a082-57d4d38f0509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd5979c1b2a4b75bf0e5afb5faf2026'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:40.103574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1055/2071] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:40 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:40.109311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.117196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1017/2072] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:39 2022] DELETE /identity/v3/users/0dd5979c1b2a4b75bf0e5afb5faf2026 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:40.121985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.122423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.122995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.134003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:40.155329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:40.156788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:40.184195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:40.186163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:40.237576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1018/2073] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:40.248016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.250221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:40.260744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9587a8-ea98-4070-8922-cb0679c5f6a4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.266149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.266896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.267191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.267473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.268175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.270327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.296172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1056/2074] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:40 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:40.300392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.312440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0027f-6ced-4654-b8a7-39f61c34360d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.318186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.319006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.319326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.319743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.320796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.324774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.334474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e19e40-f1ce-4e3f-8a77-fb111f39a15f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.336968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1057/2075] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:40.340370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.341140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.341431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.341715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.341849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.354234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.355264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60b69-f3c7-4cab-b93e-1aee20a0b4b7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.355809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.360751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.361913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.362293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.362742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] PATH_INFO: `/v3/projects/e2088410f7264f72ac96ee8bf463c966/users/abbb062335a84663abfd7abbc07de8e9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.367911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1019/2076] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:40.379904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2088410f7264f72ac96ee8bf463c966, user_id=abbb062335a84663abfd7abbc07de8e9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.383366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.401316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1058/2077] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:40 2022] PUT /identity/v3/projects/e2088410f7264f72ac96ee8bf463c966/users/abbb062335a84663abfd7abbc07de8e9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:40.408391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.443179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5af7f-d75e-42b0-8972-d26cabf1bd8a service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.549266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.550423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.550825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.551115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.552448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.558195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a26b6-d1b5-4552-8e08-684192d15fff admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.573866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1020/2078] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:40.577807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.591655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289738e8-0240-4e82-a56c-60f1f09db1a0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.596715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.597668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.597976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.598282 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] PATH_INFO: `/v3/projects/e2088410f7264f72ac96ee8bf463c966/users/abbb062335a84663abfd7abbc07de8e9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.601045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2088410f7264f72ac96ee8bf463c966, user_id=abbb062335a84663abfd7abbc07de8e9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.603787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.615494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1059/2079] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:40 2022] PUT /identity/v3/projects/e2088410f7264f72ac96ee8bf463c966/users/abbb062335a84663abfd7abbc07de8e9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:40.627646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.628194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.628590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.660751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] MFA Rules not processed for user `abbb062335a84663abfd7abbc07de8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:40.662119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:40.734358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:40.760388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1021/2080] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:40.771045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:40.773998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:40.837836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edb76b-1b78-40a2-8d14-0c1204b003d4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:40.842153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:40.843987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:40.843987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:40.843987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:40.855467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:40.857036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:40.869128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1060/2081] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:41.029603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:41.094145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:41.096227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:41.120644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b524441-3fe3-496d-98b0-b356ca066a01 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:41.126310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:41.127584 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.127970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.128459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] PATH_INFO: `/v3/users/791bed922944437280aaa84ab5eef3db` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.129882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=791bed922944437280aaa84ab5eef3db)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:41.132269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:41.218696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791bed922944437280aaa84ab5eef3db'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.230623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791bed922944437280aaa84ab5eef3db'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.255218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791bed922944437280aaa84ab5eef3db'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.256515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791bed922944437280aaa84ab5eef3db'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.259518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-78264de2-243e-40b2-847a-5a286ba44d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791bed922944437280aaa84ab5eef3db'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.259943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167368bb-2a6b-4d79-8ebe-90f678f70829 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:41.271453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:41.276216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.277985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.278279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] PATH_INFO: `/v3/projects/9ce2e059912949539a6dbc6dfa9971e0` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.290627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce2e059912949539a6dbc6dfa9971e0)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:41.290627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:41.290991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1022/2082] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:41 2022] DELETE /identity/v3/users/791bed922944437280aaa84ab5eef3db => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:41.292020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:41.295569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.295861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.296144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.326693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:41.329690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:41.365779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce2e059912949539a6dbc6dfa9971e0', 'request_id': 'req-b5f597f7-9f74-4117-8467-dabe76251716'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.376516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce2e059912949539a6dbc6dfa9971e0', 'request_id': 'req-b5f597f7-9f74-4117-8467-dabe76251716'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.383884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce2e059912949539a6dbc6dfa9971e0', 'request_id': 'req-b5f597f7-9f74-4117-8467-dabe76251716'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.395982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] The token cache is being invalidate because project 9ce2e059912949539a6dbc6dfa9971e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:41.396285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.397797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.402915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1061/2083] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:41 2022] DELETE /identity/v3/projects/9ce2e059912949539a6dbc6dfa9971e0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:41.404063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:41.405737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:41.423814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1023/2084] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:41.436128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:41.437797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:41.457818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:41.492562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f597f7-9f74-4117-8467-dabe76251716 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:41.499493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:41.500516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.500900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.501252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] PATH_INFO: `/v3/users/9207c706caca4f14a0f1f694c5954a67` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.502943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9207c706caca4f14a0f1f694c5954a67)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:41.505149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:41.514134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd6b0e-d212-4c47-80d6-4235cf18113e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:41.572489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9207c706caca4f14a0f1f694c5954a67'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.574962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:41.575714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:41.576238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:41.576582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:41.581987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9207c706caca4f14a0f1f694c5954a67'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.589467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9207c706caca4f14a0f1f694c5954a67'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.589698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:41.590156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9207c706caca4f14a0f1f694c5954a67'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.592264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:41.603618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9207c706caca4f14a0f1f694c5954a67'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:41.609210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1024/2085] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:41.610548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1062/2086] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:41 2022] DELETE /identity/v3/users/9207c706caca4f14a0f1f694c5954a67 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:42.294287 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.320337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.322700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:42.323349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2e8ac-d831-4b9c-a922-5ac04780f832 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:42.387368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:42.443207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:42.444805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.445117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.445394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] PATH_INFO: `/v3/projects/3fd02b66b878489787c3bd4b333aec99` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.446827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd02b66b878489787c3bd4b333aec99)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:42.448749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:42.501194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896b249-4de4-4a0a-aff4-3d3240df1663 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:42.516494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd02b66b878489787c3bd4b333aec99', 'request_id': 'req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.526466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd02b66b878489787c3bd4b333aec99', 'request_id': 'req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.533101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd02b66b878489787c3bd4b333aec99', 'request_id': 'req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.544758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:42.555239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] The token cache is being invalidate because project 3fd02b66b878489787c3bd4b333aec99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:42.557426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.558987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-eaf370e2-e6e7-4c12-aca6-2e1d4bf484d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.565458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1025/2087] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:42 2022] DELETE /identity/v3/projects/3fd02b66b878489787c3bd4b333aec99 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:42.568702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbf0ce-b027-4b3e-a824-ac828a63301a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.569807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbf0ce-b027-4b3e-a824-ac828a63301a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.570216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbf0ce-b027-4b3e-a824-ac828a63301a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.585291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:42.587350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.587826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.588250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] PATH_INFO: `/v3/projects/00f8e22a430c4791b1e40fe46ebab457` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.589687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f8e22a430c4791b1e40fe46ebab457)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:42.591750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:42.592914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0dfbf0ce-b027-4b3e-a824-ac828a63301a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:42.594427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0dfbf0ce-b027-4b3e-a824-ac828a63301a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:42.606223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1026/2088] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:42.615416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.616452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.616752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.643362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:42.645308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:42.658183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1027/2089] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:42.663010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.673728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f8e22a430c4791b1e40fe46ebab457', 'request_id': 'req-bee6dac4-96c7-4562-b37c-855480bab7d3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.677455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489724f8-9ddd-4b70-b92d-538f4d4b7ae2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:42.682239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f8e22a430c4791b1e40fe46ebab457', 'request_id': 'req-bee6dac4-96c7-4562-b37c-855480bab7d3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.692712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f8e22a430c4791b1e40fe46ebab457', 'request_id': 'req-bee6dac4-96c7-4562-b37c-855480bab7d3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.717144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] The token cache is being invalidate because project 00f8e22a430c4791b1e40fe46ebab457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:42.717571 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.718928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.720851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1063/2090] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:42 2022] DELETE /identity/v3/projects/00f8e22a430c4791b1e40fe46ebab457 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:42.724818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.726811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:42.786877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:42.798086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:42.800134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.800551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.800846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.804389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:42.805067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:42.807267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:42.825284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6dac4-96c7-4562-b37c-855480bab7d3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:42.830762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1028/2091] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:28:42.833417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:42.835734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.835863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.836206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.836589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] PATH_INFO: `/v3/users/abbb062335a84663abfd7abbc07de8e9` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.838060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:42.838886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbb062335a84663abfd7abbc07de8e9)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:42.841441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:42.853183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6548884-a025-436a-aac7-44ed43b2d37d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:42.865605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:42.868358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.869052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.869708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.870627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:42.873563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:42.911466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbb062335a84663abfd7abbc07de8e9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.911653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1029/2092] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:42 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:42.913743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:42.917358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbb062335a84663abfd7abbc07de8e9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.934199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbb062335a84663abfd7abbc07de8e9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.934952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbb062335a84663abfd7abbc07de8e9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.954099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b3a3fdc5-1621-462b-8693-855162054abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbb062335a84663abfd7abbc07de8e9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:42.963795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1064/2093] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:42 2022] DELETE /identity/v3/users/abbb062335a84663abfd7abbc07de8e9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:42.983691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3bbe3-467f-4ce8-a869-aa7494c16c45 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:42.984003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3bbe3-467f-4ce8-a869-aa7494c16c45 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:42.984254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3bbe3-467f-4ce8-a869-aa7494c16c45 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:42.985886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1065/2094] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 00:28:43.000727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bd330-7ac8-48fb-9dde-af08c8a65300 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.024592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.024821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.025270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.049341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] MFA Rules not processed for user `2add2dc7d84c497a937723b3badcc881`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:43.050694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.107771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.108615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.109031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.109406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.110558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.113861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.126279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.143774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1030/2095] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:42 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:43.148633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.149026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1066/2096] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:43.151003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.155355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.156227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.171507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adfa2b-81ac-4935-890d-ceb880dcb8df admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.172073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76d708-e412-4c3f-81e7-b00f9019cf9e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.177580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.177817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.178682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.178798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.179037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.179141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.179345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.179458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] PATH_INFO: `/v3/users/01c74d394beb4d7286563fc4a9687d50` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.180243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.180780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c74d394beb4d7286563fc4a9687d50)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.181932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.186638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.213661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1067/2097] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:43.216150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.219480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.241399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283323e5-516c-4594-a4b4-60f77322d5c3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.248412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.249419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.250096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.250576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.253714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c74d394beb4d7286563fc4a9687d50'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.261812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.263689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.267308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c74d394beb4d7286563fc4a9687d50'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.278206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1068/2098] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:43.279130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.283906 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c74d394beb4d7286563fc4a9687d50'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.285012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c74d394beb4d7286563fc4a9687d50'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.285992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c74d394beb4d7286563fc4a9687d50'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.293237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6233-8613-44e1-b9ec-db0614af265e service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.301044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.301944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.302312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.302670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.303714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.304393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1031/2099] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:43 2022] DELETE /identity/v3/users/01c74d394beb4d7286563fc4a9687d50 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:43.307030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.350552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1069/2100] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:43.354664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.356522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.365037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.371519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.409360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbf455-d4c8-421a-80ea-ab1e572c779e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.496249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.502764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1fc06c-f651-4cca-808b-da2fe392eeeb admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.512019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.514235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.514585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.514815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] PATH_INFO: `/v3/users/9e68823882b94b31943fa0047a2ffeba` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.516820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e68823882b94b31943fa0047a2ffeba)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.519104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.537632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.538827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.539209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.539710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] PATH_INFO: `/v3/projects/a22a17533bc5464486dc43760bd7669c/users/88fcef5324004504b119797721d1d7d9/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.561554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a17533bc5464486dc43760bd7669c, user_id=88fcef5324004504b119797721d1d7d9, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.565626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.600699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1070/2101] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:43 2022] PUT /identity/v3/projects/a22a17533bc5464486dc43760bd7669c/users/88fcef5324004504b119797721d1d7d9/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:43.602846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.606043 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e68823882b94b31943fa0047a2ffeba'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.608827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.622820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e68823882b94b31943fa0047a2ffeba'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.653154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.653154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e68823882b94b31943fa0047a2ffeba'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.658772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e68823882b94b31943fa0047a2ffeba'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.659476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-d681c6dd-c079-4bc4-a049-78d23a9ebedd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e68823882b94b31943fa0047a2ffeba'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.682623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1032/2102] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:43 2022] DELETE /identity/v3/users/9e68823882b94b31943fa0047a2ffeba => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:43.683959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.684392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.684793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.742265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] MFA Rules not processed for user `88fcef5324004504b119797721d1d7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:43.742265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.750928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8320100-e490-4cfc-8240-f94a56807a48 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.818006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.818921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.820139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.820359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] PATH_INFO: `/v3/projects/e2088410f7264f72ac96ee8bf463c966` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.821526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2088410f7264f72ac96ee8bf463c966)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.823449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.825591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1033/2103] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:43.829909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.831271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.845178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a6d7b-efe3-4876-b449-02406cd0706a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:43.849805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:43.851167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.851451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.851727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] PATH_INFO: `/v3/projects/7e7ced4b21544bd689b6b0cae01566b6` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.852951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7ced4b21544bd689b6b0cae01566b6)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:43.855006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:43.874788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2088410f7264f72ac96ee8bf463c966', 'request_id': 'req-7a2b5962-de8d-4b9c-8ac4-5717747396dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.880374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2088410f7264f72ac96ee8bf463c966', 'request_id': 'req-7a2b5962-de8d-4b9c-8ac4-5717747396dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.884761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:43.889945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2088410f7264f72ac96ee8bf463c966', 'request_id': 'req-7a2b5962-de8d-4b9c-8ac4-5717747396dc'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.903103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] The token cache is being invalidate because project e2088410f7264f72ac96ee8bf463c966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:43.903401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.904050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-7a2b5962-de8d-4b9c-8ac4-5717747396dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.904672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7ced4b21544bd689b6b0cae01566b6', 'request_id': 'req-763a5fde-a9c8-494e-8ad5-80e213025d1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.905630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1071/2104] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:43 2022] DELETE /identity/v3/projects/e2088410f7264f72ac96ee8bf463c966 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:43.906938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814199ea-770d-4343-9c87-f5af01b7e79d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.907232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814199ea-770d-4343-9c87-f5af01b7e79d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.907510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814199ea-770d-4343-9c87-f5af01b7e79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:43.911888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7ced4b21544bd689b6b0cae01566b6', 'request_id': 'req-763a5fde-a9c8-494e-8ad5-80e213025d1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.919569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7ced4b21544bd689b6b0cae01566b6', 'request_id': 'req-763a5fde-a9c8-494e-8ad5-80e213025d1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.925195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-814199ea-770d-4343-9c87-f5af01b7e79d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:43.926450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-814199ea-770d-4343-9c87-f5af01b7e79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.934100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] The token cache is being invalidate because project 7e7ced4b21544bd689b6b0cae01566b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:43.934398 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.935053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:43.936617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1034/2105] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:43 2022] DELETE /identity/v3/projects/7e7ced4b21544bd689b6b0cae01566b6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:43.941372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:43.941372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:43.991737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1072/2106] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:43.994261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328f79-7dc0-4712-8ac0-6519f71f86fe None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:43.994618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328f79-7dc0-4712-8ac0-6519f71f86fe None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:43.994975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50328f79-7dc0-4712-8ac0-6519f71f86fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.003782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a5fde-a9c8-494e-8ad5-80e213025d1c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.009519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.010190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.010475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.010754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.011452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.013457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.013752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-50328f79-7dc0-4712-8ac0-6519f71f86fe None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.015005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-50328f79-7dc0-4712-8ac0-6519f71f86fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.016802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:44.023885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1073/2107] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.025259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bae86-aa1f-43ee-a947-971ed51fae71 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.025556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bae86-aa1f-43ee-a947-971ed51fae71 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.025842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bae86-aa1f-43ee-a947-971ed51fae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.046605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1035/2108] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:43 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:44.047495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.050567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.056868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8a2bae86-aa1f-43ee-a947-971ed51fae71 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.058142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8a2bae86-aa1f-43ee-a947-971ed51fae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.124323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1074/2109] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.125902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee20d-e52b-415e-b6f8-15f810e3490d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.126252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee20d-e52b-415e-b6f8-15f810e3490d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.126700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee20d-e52b-415e-b6f8-15f810e3490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.127403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dd146-3af3-4491-83c4-897bbf36d389 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.137542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.138259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.138757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.139082 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.153310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.155249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.159453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-214ee20d-e52b-415e-b6f8-15f810e3490d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.160807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-214ee20d-e52b-415e-b6f8-15f810e3490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.167485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1036/2110] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:44.168387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.172134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1075/2111] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.174039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.174470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.176212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.180405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e051db-a8a9-4d06-9999-ab4ec7761228 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.186160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.186789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.187083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.187370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.188805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.190917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.206780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.208629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.222791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1037/2112] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/users => generated 300 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:44.222791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.228235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1076/2113] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.229793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.236045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a59fd7-3dec-4efe-926a-53decf7afb03 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.242757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.243506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.243720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.243838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1bda6-7f10-419a-8887-c2c16db24280 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.244114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.245949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:44.246094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.247693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.250303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.250857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.251145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.251426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.252266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.254588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.261727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1038/2114] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:44.262670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.270819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1077/2115] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:44.272236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.274266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35f10e-b825-4745-8519-4ab9ae0d284a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.278898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.279559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.279844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.280124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.280853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:44.281195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.282788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.285192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7558d9-91c5-4e7a-971a-8a79826b88fd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.292178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.293436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.293436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.293827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.294658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.295985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1039/2116] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:44.296858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.300236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.308848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6f2c9-685d-4470-9ea2-ff8a9ba88d8b admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.313672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.314554 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.314840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.315175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] PATH_INFO: `/v3/projects/f97b6edab9584883a0d2cc21666932e2/users/041b96109cab429e9723277d65038cb5/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.332357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97b6edab9584883a0d2cc21666932e2, user_id=041b96109cab429e9723277d65038cb5, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.332628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1078/2117] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:44.333872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.335610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.351202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1040/2118] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:44 2022] PUT /identity/v3/projects/f97b6edab9584883a0d2cc21666932e2/users/041b96109cab429e9723277d65038cb5/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:44.352363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.385649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:44.419256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51b336-f130-45b5-8d11-99365aac26ee admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.419957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754f626-3b17-4dee-858e-30c3e8f21ef1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.424828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.425258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.425614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.425902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.425997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.426182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.426278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.426493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.426885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.427514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.430457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.438609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.459085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1041/2119] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:44.460448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.471589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32f58e-63f1-43a3-9444-67550e251ce6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.476662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.477363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.477646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.477927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.478705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.480627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.486499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:44.492264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1042/2120] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:44.492511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1079/2121] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:44.494113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.497538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.514059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0382cf8-69ec-468c-936a-8d2199d09ed6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.586736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4ae49-7d2f-4317-a840-52e52b4cc517 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.591390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.591522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.592067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.592188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.592358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.592448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.592644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.592749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.593378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.593468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.595164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.595266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.605900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1080/2122] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:44.606776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.617103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1043/2123] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:44.617721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.619368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.620745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c2598-316c-480e-9fdb-922e8aa97dcc admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.625609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.626560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.626852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.627134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] PATH_INFO: `/v3/projects/f97b6edab9584883a0d2cc21666932e2/users/041b96109cab429e9723277d65038cb5/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.629168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97b6edab9584883a0d2cc21666932e2, user_id=041b96109cab429e9723277d65038cb5, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.631695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe1035-ea90-468a-b760-2a49fc9bb7a1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.631975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.636514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.637350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.637633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.637907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] PATH_INFO: `/v3/projects/1e9a5c7725ce472b9dfcb939ce742fd3` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.639073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9a5c7725ce472b9dfcb939ce742fd3)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.641019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.643738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1081/2124] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:44 2022] PUT /identity/v3/projects/f97b6edab9584883a0d2cc21666932e2/users/041b96109cab429e9723277d65038cb5/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:44.644558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.699173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9a5c7725ce472b9dfcb939ce742fd3', 'request_id': 'req-29fd5d61-fc22-43d7-a9f9-255764cba475'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:44.716282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9a5c7725ce472b9dfcb939ce742fd3', 'request_id': 'req-29fd5d61-fc22-43d7-a9f9-255764cba475'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:44.722350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9a5c7725ce472b9dfcb939ce742fd3', 'request_id': 'req-29fd5d61-fc22-43d7-a9f9-255764cba475'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:44.729333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f807afc-00d0-4d87-81b3-53b7ff636d1d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.738381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] The token cache is being invalidate because project 1e9a5c7725ce472b9dfcb939ce742fd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:44.739263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:44.739983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:44.746380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1044/2125] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:44 2022] DELETE /identity/v3/projects/1e9a5c7725ce472b9dfcb939ce742fd3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:44.747281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.748651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.789234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:44.802068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.802580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.802864 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.803169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.805561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.807390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc89187-64a0-4609-96da-263670991a8f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.824124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1082/2126] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:44.826583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811560f-b33b-48df-ab36-607ea65c2dfa None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.827093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811560f-b33b-48df-ab36-607ea65c2dfa None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.827709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811560f-b33b-48df-ab36-607ea65c2dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.850017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd5d61-fc22-43d7-a9f9-255764cba475 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.856010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.856694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.857030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.860950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.861802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.863745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9684ca-52a4-4c1c-8770-903392281a28 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.872023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e811560f-b33b-48df-ab36-607ea65c2dfa None None] MFA Rules not processed for user `041b96109cab429e9723277d65038cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.874136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e811560f-b33b-48df-ab36-607ea65c2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.876221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1045/2127] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:44.877777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.878230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.878512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.895515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:44.905471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:44.909297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.923431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1046/2128] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.924407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.925939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:44.938476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5f795-9e06-4247-b790-e56f5ddf18d3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:44.943685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:44.944908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.945162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.945510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] PATH_INFO: `/v3/projects/7e1061843a124697a5f3ddd13038a5d4/users/13eb61dec14b40449ee4bb39549c6330/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:44.961652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1061843a124697a5f3ddd13038a5d4, user_id=13eb61dec14b40449ee4bb39549c6330, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:44.964361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:44.980346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1047/2129] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:44 2022] PUT /identity/v3/projects/7e1061843a124697a5f3ddd13038a5d4/users/13eb61dec14b40449ee4bb39549c6330/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:44.981305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:44.992949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1083/2130] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:44.994309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:44.994604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:44.994888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.012697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.013925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.048657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75fb6-d69e-42ba-8391-015433adaba6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.053719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.054414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.054682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.054970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.055700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.057356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b0834-f64d-4bea-9c02-2278d13c47e9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.067680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1048/2131] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:45.069072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.069356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.069642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.075347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1084/2132] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.076475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.078055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.097433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] MFA Rules not processed for user `13eb61dec14b40449ee4bb39549c6330`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.098997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.152491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcfb42-f390-4e99-82f5-0954b1a683c6 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.156155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.156814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.157138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.157423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.179685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1049/2133] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.180770 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.188598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:45.191507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca72e4-e22a-47a5-b879-8965e3aa650f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.197074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.198023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.198380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.198663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] PATH_INFO: `/v3/projects/2d5d56c0c6444b23a1ac6cc2fcb0bb52/users/535a03ce19d346d59e6a6f6befb9dea0/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.213416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5d56c0c6444b23a1ac6cc2fcb0bb52, user_id=535a03ce19d346d59e6a6f6befb9dea0, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.216058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.222053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.223678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.231904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1050/2134] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:45 2022] PUT /identity/v3/projects/2d5d56c0c6444b23a1ac6cc2fcb0bb52/users/535a03ce19d346d59e6a6f6befb9dea0/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:45.232836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.235005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1085/2135] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.235874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.237218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.250176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a54623-996c-4066-90a9-485d7534f548 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.293198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:45.313458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.314127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.314417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.314827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.320836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83866737-0a1b-4b39-98d0-d6e613188ad3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.328262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.329136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.329756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.329756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.330592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:45.331032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.333100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe3602d-2bbc-47a4-8981-747627745bc8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.353274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1051/2136] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:45.354736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.355074 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.355366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.375214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.376450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.385603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.388633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1052/2137] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.388633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.388826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.389267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.398316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1086/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.399151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.401125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa432416-9924-4981-9c9b-48d60c839ef3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.406329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.407144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.407593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.407880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] PATH_INFO: `/v3/users/88fcef5324004504b119797721d1d7d9` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.409186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fcef5324004504b119797721d1d7d9)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.411478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.415749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057c731-0659-4c0a-8c41-3ce5c531b7e3 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.420518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.421240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.421532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.430627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.430627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.430627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.466036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1087/2139] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:45.467166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.468777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.486857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fcef5324004504b119797721d1d7d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.494563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fcef5324004504b119797721d1d7d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.508980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fcef5324004504b119797721d1d7d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.509666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fcef5324004504b119797721d1d7d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.510684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fcef5324004504b119797721d1d7d9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.524778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1053/2140] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:45 2022] DELETE /identity/v3/users/88fcef5324004504b119797721d1d7d9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:45.526000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.527556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.533394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528433e9-3ae2-4937-b8e8-9f6dcb122fb0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.542829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:45.598200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.598958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.599299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.599637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.601630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.617064 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ce5ef-0a7b-4525-93fa-24fd0ebb5911 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.621813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.622695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.622981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.623258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] PATH_INFO: `/v3/users/13eb61dec14b40449ee4bb39549c6330` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.624469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13eb61dec14b40449ee4bb39549c6330)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.626429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.668442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.669279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:45.670158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.675759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13eb61dec14b40449ee4bb39549c6330'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.680514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1088/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.684675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13eb61dec14b40449ee4bb39549c6330'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.685896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.687452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.699418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5991b-d127-43cb-ad40-299e04f06a30 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.702267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13eb61dec14b40449ee4bb39549c6330'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.703345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13eb61dec14b40449ee4bb39549c6330'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.704466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-24a7a941-5f0c-4f8f-b172-5979d69e97e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13eb61dec14b40449ee4bb39549c6330'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:45.707405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.708093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.708383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.708659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.709379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.711967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c26ca-65fe-4108-9041-790eab9aaeaf admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.721482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1054/2142] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:45 2022] DELETE /identity/v3/users/13eb61dec14b40449ee4bb39549c6330 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:45.722811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.723243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.723626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.741765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1089/2143] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:45.742935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.743468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97788d-6105-46f9-948c-e6fab606041a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.743761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97788d-6105-46f9-948c-e6fab606041a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.744074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97788d-6105-46f9-948c-e6fab606041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.744163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.763656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-de97788d-6105-46f9-948c-e6fab606041a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:45.765031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-de97788d-6105-46f9-948c-e6fab606041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.818234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1055/2144] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.821758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.823447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.838399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb356b6-d439-43b1-a6b5-aa744ac05262 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.842067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1090/2145] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:45.843045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97788d-6105-46f9-948c-e6fab606041a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.847357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.848076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.848428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.848639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.849416 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.852020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.852398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-de97788d-6105-46f9-948c-e6fab606041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.864333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1056/2146] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:45.871018 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.877371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:45.926722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:45.935267 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97788d-6105-46f9-948c-e6fab606041a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.940133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.940986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.941518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.941988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.953271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e9239-e9b2-4940-aee4-556ad62d6213 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.954403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.956458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.958254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.958983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.959382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.959734 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.970565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.971051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1091/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.971926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:45.976300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:45.984833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef05587-8020-4519-aedf-31411f7c77b6 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:45.986116 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1057/2148] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:45.990153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:45.990969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:45.991321 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:45.991959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:45.992517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:45.994551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314568c-1874-4002-add1-44c884e09956 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.009705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1092/2149] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:46.010905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.022874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9b2c-7bb0-4398-91aa-3c7be7cb0af8 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.027674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.028714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.029149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.029568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] PATH_INFO: `/v3/projects/1cad0363e07d4705bb3eccdc49a9f510/users/93caf792846c4bf3985ff9157664ecbd/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.047244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cad0363e07d4705bb3eccdc49a9f510, user_id=93caf792846c4bf3985ff9157664ecbd, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.050224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.070184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1058/2150] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:46 2022] PUT /identity/v3/projects/1cad0363e07d4705bb3eccdc49a9f510/users/93caf792846c4bf3985ff9157664ecbd/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:46.081266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.081916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.083441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.125752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] MFA Rules not processed for user `93caf792846c4bf3985ff9157664ecbd`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:46.127154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:46.285191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1093/2151] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:46.289818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.324194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:46.387496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7931ba9-6e47-4812-b3e8-becc4bcbf16c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.393399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.394572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.394934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.395276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.396169 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.399079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.431666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1059/2152] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:46 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:46.435776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.459432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96653c61-6265-4780-bb0d-fe269a28cd40 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.534107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.535953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:46.560942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.569951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.571329 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.571802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.572774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.576774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.605427 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1094/2153] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:46 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:46.609935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.628662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b7206-8510-4e8c-805b-9f5d0af3da85 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.635758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.637219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.637556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.637984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.639156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.642823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.661772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1095/2154] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:46.668869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.680331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d93bda-e1f8-413c-8306-4871ee24c2bf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.690992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831d512-f930-44c9-8f56-cafeb8b313e4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.701037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.702460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.707083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.707083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] PATH_INFO: `/v3/projects/2b4083348bfa42718631d526f6e7295f/users/90e61766aeaa4589984d6420cdd7cfa0/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.719143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.720146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.720429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.720754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] PATH_INFO: `/v3/projects/a22a17533bc5464486dc43760bd7669c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.722592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22a17533bc5464486dc43760bd7669c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.724527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.761752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4083348bfa42718631d526f6e7295f, user_id=90e61766aeaa4589984d6420cdd7cfa0, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.770086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:46.804928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1096/2155] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:46 2022] PUT /identity/v3/projects/2b4083348bfa42718631d526f6e7295f/users/90e61766aeaa4589984d6420cdd7cfa0/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:46.804928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.808488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:46.821397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22a17533bc5464486dc43760bd7669c', 'request_id': 'req-9a909502-b979-45c5-8681-5a998f50c253'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:46.832548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22a17533bc5464486dc43760bd7669c', 'request_id': 'req-9a909502-b979-45c5-8681-5a998f50c253'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:46.868462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:46.882269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22a17533bc5464486dc43760bd7669c', 'request_id': 'req-9a909502-b979-45c5-8681-5a998f50c253'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:46.912087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] The token cache is being invalidate because project a22a17533bc5464486dc43760bd7669c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:46.912489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:46.914048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:46.933278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1060/2156] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:46 2022] DELETE /identity/v3/projects/a22a17533bc5464486dc43760bd7669c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:46.934718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:46.936729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:46.978860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed226f09-6ed4-4cf8-a0aa-eaee7e8ad16d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:46.986989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:46.988917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:46.989353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:46.989807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] PATH_INFO: `/v3/projects/7e1061843a124697a5f3ddd13038a5d4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:46.992389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1061843a124697a5f3ddd13038a5d4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:46.993065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:46.995227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.040261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909502-b979-45c5-8681-5a998f50c253 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.056422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1061843a124697a5f3ddd13038a5d4', 'request_id': 'req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.062989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1061843a124697a5f3ddd13038a5d4', 'request_id': 'req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.074448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1061843a124697a5f3ddd13038a5d4', 'request_id': 'req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.090231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] The token cache is being invalidate because project 7e1061843a124697a5f3ddd13038a5d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:47.090702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.091562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.093323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1097/2157] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:46 2022] DELETE /identity/v3/projects/7e1061843a124697a5f3ddd13038a5d4 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:47.094244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.095704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.135595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.136291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.136576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.136854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.137710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.140722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d895bbb-dbb2-42b9-aba4-aafc189ae759 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.161244 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1061/2158] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:47.163863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302127c-db83-4635-8b05-b2c09449312a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.164974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302127c-db83-4635-8b05-b2c09449312a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.165710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302127c-db83-4635-8b05-b2c09449312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.189828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68bed7-9146-4673-8fdb-16dd25e1e2b2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.197885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d302127c-db83-4635-8b05-b2c09449312a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:47.199441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d302127c-db83-4635-8b05-b2c09449312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.201212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.202363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.202907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.203402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] PATH_INFO: `/v3/users/041b96109cab429e9723277d65038cb5` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.205300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] RBAC: Authorizing `identity:delete_user(user_id=041b96109cab429e9723277d65038cb5)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.207477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.208164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1062/2159] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:47.213747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302127c-db83-4635-8b05-b2c09449312a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.217412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d302127c-db83-4635-8b05-b2c09449312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.238138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302127c-db83-4635-8b05-b2c09449312a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.249377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.250570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.251725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.252132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] PATH_INFO: `/v3/projects/2b4083348bfa42718631d526f6e7295f/users/90e61766aeaa4589984d6420cdd7cfa0/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.255512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b4083348bfa42718631d526f6e7295f, user_id=90e61766aeaa4589984d6420cdd7cfa0, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.259685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f3d20-264d-4e26-a1d5-a4e0c4382595 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.285072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1063/2160] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:47 2022] PUT /identity/v3/projects/2b4083348bfa42718631d526f6e7295f/users/90e61766aeaa4589984d6420cdd7cfa0/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:47.285072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.285072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.285072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.296031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041b96109cab429e9723277d65038cb5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.301727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:47.303099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.310307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041b96109cab429e9723277d65038cb5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.312435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:47.330074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041b96109cab429e9723277d65038cb5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.332528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041b96109cab429e9723277d65038cb5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.354837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ba4290ca-afb6-401f-b790-4977ed6c472c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041b96109cab429e9723277d65038cb5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.372363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1098/2161] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:47 2022] DELETE /identity/v3/users/041b96109cab429e9723277d65038cb5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:47.382958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.383425 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.384441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.431360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1064/2162] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:47.435721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.442621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] MFA Rules not processed for user `90e61766aeaa4589984d6420cdd7cfa0`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:47.445835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.456579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eca24-d0aa-4491-95bb-ebc981e5af0a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.614711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.615390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.616303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.616585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.617551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:47.617982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.620346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.628419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1099/2163] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:47.629742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.632738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.637598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:47.641509 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1065/2164] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:28:47.645487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.653445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87eff9a-21ea-4883-9b35-43a9760b55cd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.660052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107bf75-1d0e-42ab-b600-4e4a7ca4cb5b admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.663397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.665470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.665933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.666938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] PATH_INFO: `/v3/users/fa42c494071b4e328c13d15bea0a6ff7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.667437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.668346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.668772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.669293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.669798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa42c494071b4e328c13d15bea0a6ff7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.670129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:47.672026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.672317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:47.715560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1066/2165] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:47.716566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.722088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.763838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa42c494071b4e328c13d15bea0a6ff7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.797577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa42c494071b4e328c13d15bea0a6ff7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.824811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa42c494071b4e328c13d15bea0a6ff7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.826401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa42c494071b4e328c13d15bea0a6ff7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.855326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa42c494071b4e328c13d15bea0a6ff7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:47.868838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1100/2166] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:47 2022] DELETE /identity/v3/users/fa42c494071b4e328c13d15bea0a6ff7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:47.871388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:47.876358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32ddc7-f873-4b03-85a2-0c3aaee22293 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:47.882957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:47.934073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:47.949995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:47.950589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:47.953171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:47.954246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:47.961533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.045935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb51287-1962-4b6e-b8c0-a7035ebf9c8c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.063020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.063896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.070106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.073604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.074922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.079146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d332733-906d-47b2-b93d-9a898e576e72 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.164723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1101/2167] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:47 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:48.170081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.173718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.174050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.174834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.176560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.217958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:48.224401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.225805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1067/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:48.226822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.228311 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.243988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1102/2169] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:48.248676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.261782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb930528-d8a2-44e2-8ff3-a32025d72e9f service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.273574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac2b9c-cabb-46d5-a1f8-624d9b5c43c1 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.276086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.277162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.277454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.277738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] PATH_INFO: `/v3/projects/f97b6edab9584883a0d2cc21666932e2` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.279526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97b6edab9584883a0d2cc21666932e2)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.283447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.285102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.285814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.286104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.286407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.287255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.289765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.310001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1103/2170] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:48.312725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.316382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.383050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:48.403051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97b6edab9584883a0d2cc21666932e2', 'request_id': 'req-9257daa2-c23c-433d-a674-0c53962ba255'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:48.411899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97b6edab9584883a0d2cc21666932e2', 'request_id': 'req-9257daa2-c23c-433d-a674-0c53962ba255'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:48.420363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97b6edab9584883a0d2cc21666932e2', 'request_id': 'req-9257daa2-c23c-433d-a674-0c53962ba255'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:48.441339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e74901-43ab-436c-aad4-ca4334e8fa36 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.445401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] The token cache is being invalidate because project f97b6edab9584883a0d2cc21666932e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:48.445719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:48.447141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:48.452118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1068/2171] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:48 2022] DELETE /identity/v3/projects/f97b6edab9584883a0d2cc21666932e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:48.452118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.452988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.552342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257daa2-c23c-433d-a674-0c53962ba255 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.560181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.560982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.561347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.561664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.562466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.564388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20649ce7-b2ff-41f3-b128-c7242d03b75d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.580849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1069/2172] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:48.582129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.583313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.583476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.583913 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.584179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.586545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.586933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.589435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.616994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:48.618148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.618529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.621626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1122a37-e4f7-40be-817e-4b8a90b0dc39 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.628961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1070/2173] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:48.630181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.642451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc74f0-68c7-4de1-a486-d00f7d0d05e0 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.650285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1104/2174] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:48.651652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.653117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf3e17-f929-41b9-83f5-e70550556705 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.653706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.653986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.654260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] PATH_INFO: `/v3/projects/428475ac58bb447ab64aeb93b2cb93b0/users/b53757956b874424bfbb42dcdfb4adb4/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.655351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf3e17-f929-41b9-83f5-e70550556705 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.655636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf3e17-f929-41b9-83f5-e70550556705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.698878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428475ac58bb447ab64aeb93b2cb93b0, user_id=b53757956b874424bfbb42dcdfb4adb4, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.702081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.703102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5adf3e17-f929-41b9-83f5-e70550556705 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:48.716279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5adf3e17-f929-41b9-83f5-e70550556705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.740321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1071/2175] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:48 2022] PUT /identity/v3/projects/428475ac58bb447ab64aeb93b2cb93b0/users/b53757956b874424bfbb42dcdfb4adb4/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:48.746445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1105/2176] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:48.747498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.751629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.751854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.752067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.827515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] MFA Rules not processed for user `b53757956b874424bfbb42dcdfb4adb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:48.830009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:48.915530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da504-4823-4b93-a96f-72493da866d4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.928565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:48.929458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:48.929738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:48.930047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:48.930809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:48.931130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:48.936860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:48.944359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:48.965679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1072/2177] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:48.972635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.983745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1106/2178] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:48.991973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072bcfa-f472-47f1-9cf5-86360592c9e5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:48.994657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:48.996227 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:49.005676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.013300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.013687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.013973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.014733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.021323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.086384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1073/2179] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:48 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:49.090294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:49.107038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:49.118154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae055-074a-46c4-9d1e-0325bcb5b7af None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.231407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ef2ad-3ea8-412a-97ea-1792e0d3981d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.234695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.236340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.236596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.236947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.238889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.239627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.239931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.240275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.243328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.246446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.291977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1074/2180] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:49 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:49.296189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:49.335693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.337496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.345511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf62c-8451-4312-97da-66b60892396f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.352623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1107/2181] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:49.354415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:49.355619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.356465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:49.357627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.358111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.359401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.360038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.362227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.377254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1075/2182] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:49.380845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:49.390967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825521d-94d7-45f6-9930-39381bffc48a service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.395262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804c5-fbb9-4acf-ab8e-d41deca538f2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.402786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.403678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.403967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.404246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] PATH_INFO: `/v3/projects/fab1b0b75d3f47ac9e3757ee1e0e1d05` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.406111 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab1b0b75d3f47ac9e3757ee1e0e1d05)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.408110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.414620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.418909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.421190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.421586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.424063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.425836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.458970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1076/2183] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:49.460629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:49.473440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:49.476429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab1b0b75d3f47ac9e3757ee1e0e1d05', 'request_id': 'req-6cc53c4f-4956-407f-b577-677c0119211e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:49.485301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6adaa-1482-413c-a607-d9722acb7fa6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:49.485844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab1b0b75d3f47ac9e3757ee1e0e1d05', 'request_id': 'req-6cc53c4f-4956-407f-b577-677c0119211e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:49.495851 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab1b0b75d3f47ac9e3757ee1e0e1d05', 'request_id': 'req-6cc53c4f-4956-407f-b577-677c0119211e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:49.524320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] The token cache is being invalidate because project fab1b0b75d3f47ac9e3757ee1e0e1d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:49.524621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:49.526990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6cc53c4f-4956-407f-b577-677c0119211e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:49.530262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1108/2184] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:49 2022] DELETE /identity/v3/projects/fab1b0b75d3f47ac9e3757ee1e0e1d05 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:49.535629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2d52e-fe9d-4009-a5be-43438320102f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.535917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2d52e-fe9d-4009-a5be-43438320102f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.536197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2d52e-fe9d-4009-a5be-43438320102f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.574184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-34f2d52e-fe9d-4009-a5be-43438320102f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:49.576156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-34f2d52e-fe9d-4009-a5be-43438320102f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:49.641046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:49.644789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.645746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.647896 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] PATH_INFO: `/v3/projects/e466112dd47b4f2793137d437a63d793/users/4b34655a881c486080ee4d84ed07b7b1/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.697176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1109/2185] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:49.699514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.699812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.700251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.702545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e466112dd47b4f2793137d437a63d793, user_id=4b34655a881c486080ee4d84ed07b7b1, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:49.715553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06bce1-7de7-49a1-b63a-7fc26f5e9664 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:49.727706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:49.730886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:49.742544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1077/2186] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:49 2022] PUT /identity/v3/projects/e466112dd47b4f2793137d437a63d793/users/4b34655a881c486080ee4d84ed07b7b1/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:49.753540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:49.753835 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:49.756095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:49.831165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] MFA Rules not processed for user `4b34655a881c486080ee4d84ed07b7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:49.833951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:49.984189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1110/2187] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:49.985443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.001235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8525143-f070-4662-b6ce-37d11e7693ba None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.013710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.014402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.014692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.014970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.016018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:50.016018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.024172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.030204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:50.067110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1111/2188] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:50.069415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.092676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfcb8b-32eb-47be-9a2c-801188aa6a91 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.108756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.109644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.109790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1078/2189] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:50.109990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.110283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.111078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.112825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.116553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:50.117784 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.176549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004af65-c49e-4a54-9908-96025f2a1c0b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.213271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1112/2190] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:50.213759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.214451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.214743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.215026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.215724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.218539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.219887 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.286361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1079/2191] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:50.290248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.346919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:50.383935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08044b22-7017-4ce5-aaac-e5081e990595 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.418625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca24c3b-ca08-405c-a1df-223bd4b78751 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.428583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.429815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.430247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.430662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.431543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.433787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.477288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1080/2192] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/users => generated 307 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:50.478113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.490737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0543-cab2-4d9e-b777-d625c322ea86 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.492536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.493460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.494022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.494703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.495796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.497786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.498542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.498881 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.499348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.499731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.500714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.502502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.516977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1081/2193] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:50.521146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.537054 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1113/2194] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/users => generated 286 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:50.538578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.541222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e927d7c-c8fc-4d92-8a9d-8766d6b3d4e0 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.547212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.549531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.549936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.550328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] PATH_INFO: `/v3/projects/3261e1be6412454d8f7cf16b3b62632d/users/ed764a8f09b24754a8a737b7ff6f7ae1/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.552700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd80d4-6220-4b4d-95fc-a889f878476d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.565308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.566182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.566475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.566757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.567552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.571040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.580454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3261e1be6412454d8f7cf16b3b62632d, user_id=ed764a8f09b24754a8a737b7ff6f7ae1, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.583671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.589192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1114/2195] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:50.590551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.606731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1082/2196] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:50 2022] PUT /identity/v3/projects/3261e1be6412454d8f7cf16b3b62632d/users/ed764a8f09b24754a8a737b7ff6f7ae1/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:50.610053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3944e6-c3c8-432c-a935-6e62f5a1bd82 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.610752 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.684316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.685121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.685415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.685697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.686533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.688156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.701419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1115/2197] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:50.707087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.712863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415aae3f-7eca-4d8c-84f0-34a9a8a4a706 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.718760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b3850-ecfc-4940-b266-612310ce4f08 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.721615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.722310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.722602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.722885 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.723628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.726111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.733466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.735195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.735855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.736873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] PATH_INFO: `/v3/projects/949f02f0c2cd4a2587f71b200905fea6/users/db747b81fb99431aa91e374b1aa5901f/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.737697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1083/2198] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:50.742293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.754191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d7d70-e5f4-4e4e-8f7a-055c16923bf2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:50.760098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:50.761382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.761772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.762154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] PATH_INFO: `/v3/projects/3261e1be6412454d8f7cf16b3b62632d/users/ed764a8f09b24754a8a737b7ff6f7ae1/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.762355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=949f02f0c2cd4a2587f71b200905fea6, user_id=db747b81fb99431aa91e374b1aa5901f, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.764390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3261e1be6412454d8f7cf16b3b62632d, user_id=ed764a8f09b24754a8a737b7ff6f7ae1, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:50.765144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce7a1-91a6-4d6b-a403-021feebb8a5d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.767891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3998a-feb9-4714-a79b-2d49e71e3494 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:50.781238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1084/2199] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:50 2022] PUT /identity/v3/projects/3261e1be6412454d8f7cf16b3b62632d/users/ed764a8f09b24754a8a737b7ff6f7ae1/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:50.791934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.792332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.792720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.793253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1116/2200] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:50 2022] PUT /identity/v3/projects/949f02f0c2cd4a2587f71b200905fea6/users/db747b81fb99431aa91e374b1aa5901f/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:50.795489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:50.795841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:50.796131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:50.799242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:50.824497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] MFA Rules not processed for user `ed764a8f09b24754a8a737b7ff6f7ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:50.826174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:50.831713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] MFA Rules not processed for user `db747b81fb99431aa91e374b1aa5901f`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:50.833980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:50.879579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:50.927865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1085/2201] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:50.937139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1117/2202] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:50.940234 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.941736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:50.946406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:50.947855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:51.020376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006cd31a-9f5e-4878-8824-003972ce64ce None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:51.028813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:51.030608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.031213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.032167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.034978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe403adb-4f19-48d0-b85c-70a09716d742 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:51.042641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:51.043801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.044210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.044599 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.045163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:51.046846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:51.055859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:51.057882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:51.061327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1086/2203] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:51.069379 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1118/2204] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:51.104793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:51.177676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:51.190225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932aef5-9ee8-4da6-9c72-b4d755a9ecce service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:51.199358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:51.200791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.201200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.201617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.289754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55650fe0-d59a-4758-86af-6e5a8446bad4 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:51.305306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:51.308458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.308773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.309717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] PATH_INFO: `/v3/users/db747b81fb99431aa91e374b1aa5901f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.312202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=db747b81fb99431aa91e374b1aa5901f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:51.314235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:51.318364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:51.320228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a705-8925-40d7-950c-d45aea36fa4b service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:51.346632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1119/2205] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:51.380507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:51.403142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db747b81fb99431aa91e374b1aa5901f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:51.418555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db747b81fb99431aa91e374b1aa5901f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:51.435517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db747b81fb99431aa91e374b1aa5901f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:51.436464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db747b81fb99431aa91e374b1aa5901f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:51.437361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db747b81fb99431aa91e374b1aa5901f'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:51.452801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1087/2206] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:51 2022] DELETE /identity/v3/users/db747b81fb99431aa91e374b1aa5901f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:51.457116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.457582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.458005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.484738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:51.486749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:51.606242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1120/2207] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:51.624333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:51.625911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:51.747286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afe3c7-03c9-42aa-961f-121dd1f4f71d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:51.757243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:51.760800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:51.761238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:51.761525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:51.778623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:51.780238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de71c0-90c1-463a-b672-e0d6fa954a04 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:51.806011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1088/2208] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:52.299732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:52.305213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:52.327176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9a8b-2db9-4ff9-ad68-667f3123dad8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:52.339536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:52.340440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.340727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.341110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] PATH_INFO: `/v3/projects/949f02f0c2cd4a2587f71b200905fea6` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.345787 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949f02f0c2cd4a2587f71b200905fea6)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:52.348398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:52.356487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:52.432749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949f02f0c2cd4a2587f71b200905fea6', 'request_id': 'req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:52.439600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949f02f0c2cd4a2587f71b200905fea6', 'request_id': 'req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:52.477372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949f02f0c2cd4a2587f71b200905fea6', 'request_id': 'req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:52.493449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] The token cache is being invalidate because project 949f02f0c2cd4a2587f71b200905fea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:52.493753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:52.495199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1350fa69-e06d-4ed8-854a-e1ff2dfcd249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:52.504928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.505126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.505384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.522445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1121/2209] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:52 2022] DELETE /identity/v3/projects/949f02f0c2cd4a2587f71b200905fea6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:52.543310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:52.544763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:52.678630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1089/2210] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:52.680374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62284f-1301-4982-b55d-a9d3997b2cbf None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.680710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62284f-1301-4982-b55d-a9d3997b2cbf None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.681097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62284f-1301-4982-b55d-a9d3997b2cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.719140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-fa62284f-1301-4982-b55d-a9d3997b2cbf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:52.720385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fa62284f-1301-4982-b55d-a9d3997b2cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:52.756268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1122/2211] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:52.760765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:52.788855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.789240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.789583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.794237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560ce65-1f51-4865-8ffb-9edd6a3c47a7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:52.799226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:52.799951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.800285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.800567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.801346 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:52.801663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:52.804348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:52.821025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1090/2212] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:52.822371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:52.824960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] MFA Rules not processed for user `535a03ce19d346d59e6a6f6befb9dea0`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:52.826161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:52.834120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c216a15-d9f1-49a8-9c42-0c8cfc44511d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:52.839364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:52.840115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.840476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.840867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.841711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:52.844035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:52.880799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1091/2213] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:52 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:28:52.884713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:52.909640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:52.932129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1123/2214] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:52.946570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:52.948710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:52.963954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad324b-02fb-426f-8aa7-064b859f94b1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:52.969950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:52.971110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:52.971400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:52.971681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:52.972799 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:52.975832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.002286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1092/2215] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:52 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:53.006130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:53.016549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203f88e-b136-4208-b19c-8ef778d0e622 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:53.022537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:53.023386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.023677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.023958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.024736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:53.026399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.037001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0dd6ef-d24a-48b8-b723-dfa6873a5b89 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:53.038365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1093/2216] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:53.042566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:53.051160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:53.055530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa063e0-c46d-4149-9775-9ddda6f852c4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:53.060487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:53.061315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.061502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.061790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.062539 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:53.064221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.077128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1094/2217] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:53.080536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:53.094090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86892dfd-5082-4924-be76-f4234c15313e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:53.099069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:53.100776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.101185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.101482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] PATH_INFO: `/v3/projects/84db54c90fd146a0bf9927174c26462f/users/2438c100e53f467dbec2eeac56cab950/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.118634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84db54c90fd146a0bf9927174c26462f, user_id=2438c100e53f467dbec2eeac56cab950, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:53.121501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897e744-4393-4bcd-90cd-13e8bbadfd7d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.121617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:53.122314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.122622 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.122920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.151596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1095/2218] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:53 2022] PUT /identity/v3/projects/84db54c90fd146a0bf9927174c26462f/users/2438c100e53f467dbec2eeac56cab950/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:53.154180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f936f-6504-4d6e-9318-808397bd3b0a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.154573 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f936f-6504-4d6e-9318-808397bd3b0a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.154857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f936f-6504-4d6e-9318-808397bd3b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.186702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-358f936f-6504-4d6e-9318-808397bd3b0a None None] MFA Rules not processed for user `2438c100e53f467dbec2eeac56cab950`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:53.188011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-358f936f-6504-4d6e-9318-808397bd3b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:53.196257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:53.198121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.208569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1124/2219] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:28:53.264567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1096/2220] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:53.280008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:53.281587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:53.346544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17d0e3-8064-420a-8da0-d83571688913 service nova] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:53.406087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:53.407374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:53.407374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:53.407574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:53.417712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:53.419226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:53.437022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1125/2221] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:54.673124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ad79-2500-4371-b5a1-124f5aeba7f6 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:54.673601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ad79-2500-4371-b5a1-124f5aeba7f6 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:54.673789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623ad79-2500-4371-b5a1-124f5aeba7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:54.691816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7623ad79-2500-4371-b5a1-124f5aeba7f6 None None] MFA Rules not processed for user `535a03ce19d346d59e6a6f6befb9dea0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:54.693046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7623ad79-2500-4371-b5a1-124f5aeba7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:54.750444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:54.769324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1097/2222] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:57.202597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:57.204591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:57.223724 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:57.317044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083e385-8f34-4da9-8396-90a8a47c9ee7 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:57.324338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:57.325406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:57.325785 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:57.326372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] PATH_INFO: `/v3/users/93caf792846c4bf3985ff9157664ecbd` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:57.327904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93caf792846c4bf3985ff9157664ecbd)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:57.338824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:57.433563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93caf792846c4bf3985ff9157664ecbd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:57.443138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93caf792846c4bf3985ff9157664ecbd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:57.453553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93caf792846c4bf3985ff9157664ecbd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:57.454631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93caf792846c4bf3985ff9157664ecbd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:57.472295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93caf792846c4bf3985ff9157664ecbd'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:57.484957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1126/2223] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:57 2022] DELETE /identity/v3/users/93caf792846c4bf3985ff9157664ecbd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:57.486618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:57.487069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:57.487303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:57.508409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:57.510468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:57.596907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1098/2224] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:57.606505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:57.608001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:57.723507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c69470-199d-46a7-b89f-e611e5feafd7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:57.728523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:57.730164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:57.730164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:57.731770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:57.765565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:57.767716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:57.784816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1127/2225] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:28:58.303693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:58.305309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:58.320300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546b48b-caf5-4ea6-aeef-804a3834493d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:58.326324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:58.327381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:58.327829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:58.328313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] PATH_INFO: `/v3/projects/1cad0363e07d4705bb3eccdc49a9f510` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:58.330093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cad0363e07d4705bb3eccdc49a9f510)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:58.332335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:58.416102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cad0363e07d4705bb3eccdc49a9f510', 'request_id': 'req-e51c2302-5a00-4ef6-ae74-84428d0fb49a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.423708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cad0363e07d4705bb3eccdc49a9f510', 'request_id': 'req-e51c2302-5a00-4ef6-ae74-84428d0fb49a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.434598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cad0363e07d4705bb3eccdc49a9f510', 'request_id': 'req-e51c2302-5a00-4ef6-ae74-84428d0fb49a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.451918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] The token cache is being invalidate because project 1cad0363e07d4705bb3eccdc49a9f510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:58.452436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.453323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.455391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1099/2226] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:58 2022] DELETE /identity/v3/projects/1cad0363e07d4705bb3eccdc49a9f510 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:58.459407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:58.470590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43efb80-a80e-410d-a21c-c815d7c11f52 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:58.507905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:58.541677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:58.542554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:58.542835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:58.543109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] PATH_INFO: `/v3/users/90e61766aeaa4589984d6420cdd7cfa0` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:58.544389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e61766aeaa4589984d6420cdd7cfa0)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:58.546482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:58.606255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e61766aeaa4589984d6420cdd7cfa0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.613077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e61766aeaa4589984d6420cdd7cfa0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.620906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e61766aeaa4589984d6420cdd7cfa0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.621568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e61766aeaa4589984d6420cdd7cfa0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.633647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ffbe2c8e-1e97-4af3-bf98-f87f13d0a656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e61766aeaa4589984d6420cdd7cfa0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:58.639440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1128/2227] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:28:58 2022] DELETE /identity/v3/users/90e61766aeaa4589984d6420cdd7cfa0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:58.959202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:58.961138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:58.975658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2302-5a00-4ef6-ae74-84428d0fb49a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.039280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.040149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.040430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.040705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] PATH_INFO: `/v3/projects/2b4083348bfa42718631d526f6e7295f` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.041915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b4083348bfa42718631d526f6e7295f)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.043964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.068055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:59.094854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b4083348bfa42718631d526f6e7295f', 'request_id': 'req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:59.102393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b4083348bfa42718631d526f6e7295f', 'request_id': 'req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:59.109020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b4083348bfa42718631d526f6e7295f', 'request_id': 'req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:59.126314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] The token cache is being invalidate because project 2b4083348bfa42718631d526f6e7295f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:28:59.128919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:59.128919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8898ab04-8ef2-4e6a-91ad-be68ddee9bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:28:59.129628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1100/2228] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:28:58 2022] DELETE /identity/v3/projects/2b4083348bfa42718631d526f6e7295f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:28:59.143478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.143773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.144058 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.165736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:59.166950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:59.240965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1129/2229] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:59.248627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.249042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.249397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.276971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:59.278471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:59.288743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1101/2230] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:28:59.293152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.303656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e185ef2-3dd8-4788-bbb7-e93021564156 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.308929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.309589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.309874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.310168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.312853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:28:59.312853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.312853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.328932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1130/2231] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:28:59.331520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.342109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852c4fc-51ca-4009-926c-5b714440dcb2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.347096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.347898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.348199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.348504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.349424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.352502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.383089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1102/2232] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:28:59 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:59.387256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.398543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8940d-bc35-42e8-af2c-1be6893ccfc7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.437135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:28:59.456730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.457430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.457717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.457995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.458682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.461306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.480063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1131/2233] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:28:59 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:28:59.484245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.494875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c48b6-babc-4390-a55e-3990376cf949 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.499458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.500114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.500400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.500677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.501448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.503049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.519868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1103/2234] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:28:59.524442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.534678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20220bd2-cafa-4b63-b91f-ec8b6af7b576 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.539751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.540430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.540718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.541035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.541779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.543974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a44d-9d9e-4a9a-9f00-b524bbffa395 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.554334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1132/2235] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:28:59.558446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.570362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13d4c3-7502-40c2-9fe0-3c3de0552548 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.575264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.576155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.576440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.576780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] PATH_INFO: `/v3/projects/c8055262b50d4e93b290315d0e57bd70/users/ac38b6c63c1649d68302de51cc3424c1/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.593940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8055262b50d4e93b290315d0e57bd70, user_id=ac38b6c63c1649d68302de51cc3424c1, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.596646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.610852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1104/2236] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:28:59 2022] PUT /identity/v3/projects/c8055262b50d4e93b290315d0e57bd70/users/ac38b6c63c1649d68302de51cc3424c1/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:28:59.616269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad4b95-dd2d-43ba-98ba-b326b5eed5e6 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.616770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad4b95-dd2d-43ba-98ba-b326b5eed5e6 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.617088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad4b95-dd2d-43ba-98ba-b326b5eed5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.635696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-40ad4b95-dd2d-43ba-98ba-b326b5eed5e6 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:28:59.637059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-40ad4b95-dd2d-43ba-98ba-b326b5eed5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:59.708639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1133/2237] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:28:59.719236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:28:59.721135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:28:59.779449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228d0d2-2449-469f-9086-195e9ff01d3c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:28:59.783304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:28:59.783949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:28:59.784232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:28:59.784506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:28:59.794461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:28:59.795839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:28:59.806301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1105/2238] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:00.701886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:00.702461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:00.702461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:00.738489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] MFA Rules not processed for user `ac38b6c63c1649d68302de51cc3424c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:00.739643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:00.836064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1134/2239] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:00.848786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:00.850818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:00.933830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:00.939363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03be393-59ea-4806-b13d-6b04b384eb70 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:00.946258 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:00.947324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:00.948524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:00.949476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:00.966401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:00.976501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771e02f-1aef-46be-aa5b-63da81f532e2 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:00.984288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1106/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:01.639129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:01.640616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:01.655402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee06aa2-8a5d-43ed-be2b-191c044005ee None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:01.660732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:01.662121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:01.662899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:01.663180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] PATH_INFO: `/v3/users/4b34655a881c486080ee4d84ed07b7b1` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:01.664592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b34655a881c486080ee4d84ed07b7b1)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:01.667824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:01.732417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:01.735897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b34655a881c486080ee4d84ed07b7b1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:01.746585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b34655a881c486080ee4d84ed07b7b1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:01.753941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b34655a881c486080ee4d84ed07b7b1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:01.754898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b34655a881c486080ee4d84ed07b7b1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:01.769135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b34655a881c486080ee4d84ed07b7b1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:01.777234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1135/2241] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:01 2022] DELETE /identity/v3/users/4b34655a881c486080ee4d84ed07b7b1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:01.781855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:01.782345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:01.782777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:01.800820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:01.802446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:01.867345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1107/2242] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:01.877095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:01.878555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:01.946982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03093304-b437-413e-a08a-ad94b65b1186 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:01.946982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:01.946982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:01.946982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:01.947372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:01.955731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:01.957318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349966e-8ed2-4808-acc2-d2839ae35e00 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:01.968364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1136/2243] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:02.359318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:02.361139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.384230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa5c6-8b6e-40dd-8b9e-d300d63d0933 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:02.392000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:02.393626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.394246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.394700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] PATH_INFO: `/v3/projects/e466112dd47b4f2793137d437a63d793` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:02.397058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e466112dd47b4f2793137d437a63d793)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:02.400207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:02.434547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.434737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.435020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:02.453862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] MFA Rules not processed for user `ac38b6c63c1649d68302de51cc3424c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:02.455071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.458179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e466112dd47b4f2793137d437a63d793', 'request_id': 'req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.465502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e466112dd47b4f2793137d437a63d793', 'request_id': 'req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.471982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e466112dd47b4f2793137d437a63d793', 'request_id': 'req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.486158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] The token cache is being invalidate because project e466112dd47b4f2793137d437a63d793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:02.486764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.487593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.489467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1108/2244] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:02 2022] DELETE /identity/v3/projects/e466112dd47b4f2793137d437a63d793 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:02.497909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:02.502497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.530738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1137/2245] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:02.531421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:02.532751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.576750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce843df-f11c-49e6-8832-4bcfc9ef6fac admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:02.583068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:02.584851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.585595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.585910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] PATH_INFO: `/v3/users/ed764a8f09b24754a8a737b7ff6f7ae1` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:02.587231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed764a8f09b24754a8a737b7ff6f7ae1)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:02.589204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:02.606520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:02.618284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed808a-26cd-460e-a1ce-9a52635f7c1a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:02.623663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:02.624555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.624841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.625140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] PATH_INFO: `/v3/users/b53757956b874424bfbb42dcdfb4adb4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:02.626503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53757956b874424bfbb42dcdfb4adb4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:02.628664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:02.645014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:02.650440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed764a8f09b24754a8a737b7ff6f7ae1'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.659185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed764a8f09b24754a8a737b7ff6f7ae1'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.672946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed764a8f09b24754a8a737b7ff6f7ae1'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.673705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed764a8f09b24754a8a737b7ff6f7ae1'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.674853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-99480ed2-4d37-4b5c-b6eb-62b8c52d8a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed764a8f09b24754a8a737b7ff6f7ae1'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.691562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1109/2246] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:02 2022] DELETE /identity/v3/users/ed764a8f09b24754a8a737b7ff6f7ae1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:29:02.699120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53757956b874424bfbb42dcdfb4adb4'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.713188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53757956b874424bfbb42dcdfb4adb4'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.725808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53757956b874424bfbb42dcdfb4adb4'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.725808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53757956b874424bfbb42dcdfb4adb4'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.741750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53757956b874424bfbb42dcdfb4adb4'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:02.751813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1138/2247] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:02 2022] DELETE /identity/v3/users/b53757956b874424bfbb42dcdfb4adb4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:02.756212 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.756576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.757511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:02.779738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:02.781485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.857558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1110/2248] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:02.887187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:02.888729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:02.989156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605cae3-de35-4713-9d9f-79cda2673cca admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:02.994570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:02.995261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:02.995573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:02.995856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.007084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:03.010764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae7ee5-3291-4d82-91a9-cab9e90c0314 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:03.020134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1139/2249] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:03.171367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:03.173735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.189723 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7623-7d22-4db7-8c9d-074a8e52105b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:03.198136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:03.199602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.200220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.200722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] PATH_INFO: `/v3/users/073e63a99e224eb58626484f889693ea` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.202579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=073e63a99e224eb58626484f889693ea)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:03.205589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:03.320729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073e63a99e224eb58626484f889693ea'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.328598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073e63a99e224eb58626484f889693ea'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.352982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073e63a99e224eb58626484f889693ea'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.353205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073e63a99e224eb58626484f889693ea'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.353821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073e63a99e224eb58626484f889693ea'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.372125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1111/2250] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:03 2022] DELETE /identity/v3/users/073e63a99e224eb58626484f889693ea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:03.377217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.377486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.378026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.415857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:03.418621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.441601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:03.444237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.471605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:03.594037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1140/2251] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:03.613245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433e99-32c9-427c-9fc3-e156ea4cc89f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:03.617043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:03.617993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.636255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:03.637909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.637909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.638145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] PATH_INFO: `/v3/projects/3261e1be6412454d8f7cf16b3b62632d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.639467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3261e1be6412454d8f7cf16b3b62632d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:03.641472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:03.679919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:03.730496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3261e1be6412454d8f7cf16b3b62632d', 'request_id': 'req-e7123ad1-3f85-40d9-922e-56a3f7196937'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.738142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3261e1be6412454d8f7cf16b3b62632d', 'request_id': 'req-e7123ad1-3f85-40d9-922e-56a3f7196937'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.738698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9c841-6fb7-4d25-baa7-14422d0f0a64 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:03.748503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3261e1be6412454d8f7cf16b3b62632d', 'request_id': 'req-e7123ad1-3f85-40d9-922e-56a3f7196937'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.775523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] The token cache is being invalidate because project 3261e1be6412454d8f7cf16b3b62632d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:03.776013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.778392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e7123ad1-3f85-40d9-922e-56a3f7196937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:03.783381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1112/2252] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:03 2022] DELETE /identity/v3/projects/3261e1be6412454d8f7cf16b3b62632d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:03.788181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.788689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.789145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.826840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:03.828274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.842079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:03.843415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.843896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.844180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.846238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.902869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1113/2253] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:03.903838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:03.905229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:03.920303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe99c68-53bd-42ea-ad05-7df7312d8b8a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:03.926482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:03.928060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:03.928524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:03.928865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] PATH_INFO: `/v3/projects/428475ac58bb447ab64aeb93b2cb93b0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:03.930505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] RBAC: Authorizing `identity:delete_project(project_id=428475ac58bb447ab64aeb93b2cb93b0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:03.932695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:03.972959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:03.978209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b23261-7174-409a-bae5-68872e568b0e service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:03.982898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:03.999769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428475ac58bb447ab64aeb93b2cb93b0', 'request_id': 'req-9313c9e0-7875-4c63-8069-6c5bf0e2671d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.003897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1141/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:04.006007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.006424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.006893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.029842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428475ac58bb447ab64aeb93b2cb93b0', 'request_id': 'req-9313c9e0-7875-4c63-8069-6c5bf0e2671d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.039344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428475ac58bb447ab64aeb93b2cb93b0', 'request_id': 'req-9313c9e0-7875-4c63-8069-6c5bf0e2671d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.048830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:04.052037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.063023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] The token cache is being invalidate because project 428475ac58bb447ab64aeb93b2cb93b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:04.063824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.066891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-9313c9e0-7875-4c63-8069-6c5bf0e2671d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.072646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1114/2255] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:03 2022] DELETE /identity/v3/projects/428475ac58bb447ab64aeb93b2cb93b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:04.085803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa23e-829a-4651-ba15-4fae8ee14392 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.086113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa23e-829a-4651-ba15-4fae8ee14392 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.086420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09afa23e-829a-4651-ba15-4fae8ee14392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.124124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-09afa23e-829a-4651-ba15-4fae8ee14392 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:04.125877 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-09afa23e-829a-4651-ba15-4fae8ee14392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.166529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1142/2256] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:04.171510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.188817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffafc3-c93a-497e-852c-3088cff9858d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.203361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.204305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.204757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.205099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.205928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:04.206254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.208016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.215365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1115/2257] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:04.216844 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.217168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.217578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.229794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1143/2258] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:29:04.231528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.243052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cc80-41c5-40f2-ac7b-3d53f89020f8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.250013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:04.250309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.250984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.251273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.251550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.251669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.252251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.254403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.274072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1116/2259] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:04.275892 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.288164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbad967-ba9b-4584-a2ed-0990e71158e2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.296431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1144/2260] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:04.300853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.306531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:04.307076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.307761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.308089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.308473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.309228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:04.309538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.311549 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.328820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1117/2261] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:29:04.332970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.346441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ed168-dbf9-4b2e-921a-f578c65d22ed admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.414735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ec5-5317-43ea-85b9-8a080aa5a32a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.429033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.430259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.431258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.431550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.433566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.439838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.452772 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.453533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.453824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.454200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.454921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.458048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.490912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1145/2262] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:04.490912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.495529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1118/2263] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:29:04.500409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.501730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.501863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.526781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb85b4-d4c4-44e4-a39b-5f0e707824a0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.570934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:04.635055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.635951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.636239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.636542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] PATH_INFO: `/v3/users/2438c100e53f467dbec2eeac56cab950` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.637853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2438c100e53f467dbec2eeac56cab950)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.639868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8444b46-caf0-42cb-b140-98fc2ceda6bb admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.640039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.647324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.648314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.648908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.649286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] PATH_INFO: `/v3/projects/5313813c5d2e4f01915e812d4665ad02` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.650907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5313813c5d2e4f01915e812d4665ad02)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.653943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.705153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2438c100e53f467dbec2eeac56cab950'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.723773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5313813c5d2e4f01915e812d4665ad02', 'request_id': 'req-b5ed156f-64f6-45cd-8f53-32bc9548110a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.730142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2438c100e53f467dbec2eeac56cab950'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.732400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5313813c5d2e4f01915e812d4665ad02', 'request_id': 'req-b5ed156f-64f6-45cd-8f53-32bc9548110a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.737900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2438c100e53f467dbec2eeac56cab950'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.738690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2438c100e53f467dbec2eeac56cab950'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.739784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5313813c5d2e4f01915e812d4665ad02', 'request_id': 'req-b5ed156f-64f6-45cd-8f53-32bc9548110a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.754757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] The token cache is being invalidate because project 5313813c5d2e4f01915e812d4665ad02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:04.754987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.755234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2438c100e53f467dbec2eeac56cab950'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.755641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:04.757449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1119/2264] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:04 2022] DELETE /identity/v3/projects/5313813c5d2e4f01915e812d4665ad02 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:04.761586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.762939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.772906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1146/2265] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:04 2022] DELETE /identity/v3/users/2438c100e53f467dbec2eeac56cab950 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:04.773841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.775334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.795900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:04.851726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed156f-64f6-45cd-8f53-32bc9548110a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.875742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40825593-7628-4e72-bdcb-f122f29f90d5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.881309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.881992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.882276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.882555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.883252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.885856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca579812-ca75-4d0d-8807-f84590121243 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.914068 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1147/2266] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:04.916222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f06a07-6ed3-41bf-81dc-885c3f0516ae None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.916939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f06a07-6ed3-41bf-81dc-885c3f0516ae None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.917436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f06a07-6ed3-41bf-81dc-885c3f0516ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.937506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.938425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.938825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.939278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.940093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.940680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-88f06a07-6ed3-41bf-81dc-885c3f0516ae None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:04.941890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.942059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-88f06a07-6ed3-41bf-81dc-885c3f0516ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.948647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:04.954643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1148/2267] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:04.955488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1120/2268] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:04.956842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:04.959465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.959826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.960138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.972006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71189217-6d34-4277-a035-878e31ccad81 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:04.978508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:04.979651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:04.980219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:04.980719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:04.981780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:04.981928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:04.983172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:04.984057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c333-724a-45cc-a239-10a2588f8cc1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:04.995003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1149/2269] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:04.998919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1121/2270] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:04.999872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.000305 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.000601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.000903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.002543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.015916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3770c756-1042-4bb0-a54c-a3d3b535b414 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.022179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.023043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.023353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.023751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.024683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.027579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.029735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:05.033708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.043510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1150/2271] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:05.044632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.055766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52155f-dd0b-4f70-8494-bffb636c9775 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.061246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1122/2272] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:05.061408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.061728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.062290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.062702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.063077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.063938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.065731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.066460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.078256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1151/2273] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:05.079113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.108265 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee0db9-22f7-4ff4-81b4-52031af613ab admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.122134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.123788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.124077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.124358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] PATH_INFO: `/v3/projects/5b307219f36a42afa6862612bd580f80/users/523ca2a00b684a5faf9af1c405c7ac6d/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.148759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b307219f36a42afa6862612bd580f80, user_id=523ca2a00b684a5faf9af1c405c7ac6d, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.151902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.163144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a260d3-4350-46c6-83f9-edac663cdde2 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.180704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1152/2274] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:05 2022] PUT /identity/v3/projects/5b307219f36a42afa6862612bd580f80/users/523ca2a00b684a5faf9af1c405c7ac6d/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:05.180704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.221943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.222862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.223223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.223596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.261160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08919-0d28-45bf-b5ec-594043434d9f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.266124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.266730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.267017 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.267296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.268174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:05.268492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.270214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.287739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1153/2275] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:29:05.288673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.301598 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56693f9e-378a-49e9-b259-5ba02562948f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.307446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.308362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.308664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.308968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] PATH_INFO: `/v3/projects/43eb84c5eba749dea293084a683fde29/users/5b84e1bb697b4c14a3d8133347524ccb/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.314475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.315948 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8351ab9-5041-43d8-8664-aa52e83a61b1 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.327268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1123/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:05.328755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fb21-f78f-4622-a7a7-9cfad993709b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.329342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fb21-f78f-4622-a7a7-9cfad993709b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.329637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fb21-f78f-4622-a7a7-9cfad993709b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.330131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43eb84c5eba749dea293084a683fde29, user_id=5b84e1bb697b4c14a3d8133347524ccb, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.335101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.354374 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1154/2277] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:05 2022] PUT /identity/v3/projects/43eb84c5eba749dea293084a683fde29/users/5b84e1bb697b4c14a3d8133347524ccb/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:05.355199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.371103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4850fb21-f78f-4622-a7a7-9cfad993709b None None] MFA Rules not processed for user `523ca2a00b684a5faf9af1c405c7ac6d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:05.372338 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:05.373682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4850fb21-f78f-4622-a7a7-9cfad993709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.426096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:05.471231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bece1-6dc7-49a6-8645-7d860120c1c6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.479400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.480584 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.481163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.481655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.482629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.483108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1124/2278] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:05.484917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.486533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.487060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.491100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.529426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] MFA Rules not processed for user `5b84e1bb697b4c14a3d8133347524ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:05.531190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.540331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1155/2279] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:05 2022] POST /identity/v3/projects => generated 363 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:05.541277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.666270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19429e-b00d-45e2-bcab-0762593bedbf admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.676086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1125/2280] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:05.676366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.676472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.676561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.676561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.676561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.676561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.678089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.699746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921fefd-d3da-4838-b138-33ac98c5af18 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.713023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.713914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.714292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.714664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.715666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.718687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.748869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1156/2281] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:05 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:05.749896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.760264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.764933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1126/2282] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:05 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:05.765785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:05.773960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:05.873376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3641c-39d9-489c-9932-6dfd7b21c258 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.903669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c6cc6-8c09-46ac-a177-7fc4497a37e2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:05.918661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:05.919922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:05.921372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:05.921893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] PATH_INFO: `/v3/projects/84db54c90fd146a0bf9927174c26462f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:05.923855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84db54c90fd146a0bf9927174c26462f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:05.926840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:05.990218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:06.010484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.011843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.012338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.012839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.029548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84db54c90fd146a0bf9927174c26462f', 'request_id': 'req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:06.038079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84db54c90fd146a0bf9927174c26462f', 'request_id': 'req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:06.043595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:06.050692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84db54c90fd146a0bf9927174c26462f', 'request_id': 'req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:06.071484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] The token cache is being invalidate because project 84db54c90fd146a0bf9927174c26462f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:06.072264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:06.073249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:06.075677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1157/2283] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:05 2022] DELETE /identity/v3/projects/84db54c90fd146a0bf9927174c26462f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:06.076690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.086315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:06.101809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.112125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.135874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1127/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:06.138736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.161384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436adc4-7eff-485d-a49d-ec388b18e02c service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.358668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b1eac-0a30-4c83-8bff-793fd03d6a6f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.369988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.371180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.371616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.372099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.374917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.377610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.378641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.379028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.379407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.380651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.382556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.382741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.397983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1128/2285] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:06.399614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.402759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:06.429696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac7475-1a83-4467-9a8a-40cdfa9f212f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.435575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.436368 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.436758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.439890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1158/2286] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:06 2022] POST /identity/v3/users => generated 289 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:06.440021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.440021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.441138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.441572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.454229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838384a-4e53-426c-bb3a-dc2fecc887b1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.457682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1129/2287] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:06.461652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.469378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.470703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.471808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.471808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.472035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.473731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.508063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8bcd7-22d3-4ead-a298-050e1e91e930 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.511919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1159/2288] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:06.513476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.538507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.539320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.539320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.542101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867d3a-7b94-4721-842c-6c4e4c61d884 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.545637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] PATH_INFO: `/v3/projects/9696bd2b296b4aed865245217d79f2f6/users/5379b6cf2e004c16bc62171b98e7e7f0/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.554812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.555643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.555912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.557208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.557983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.560368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.593215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1160/2289] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:29:06.595552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.608450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9696bd2b296b4aed865245217d79f2f6, user_id=5379b6cf2e004c16bc62171b98e7e7f0, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.611231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7baa7-2f19-4f13-b849-6c1969c9c0d7 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.612364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e4ed1-7718-45e8-b114-8e782542411f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:06.626992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:06.639042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.639042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.639042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] PATH_INFO: `/v3/projects/17d79ebf426b4ca8b5571815460ac2e7/users/0a1fe8eea9cb472784b71f7a37ca7b57/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.645153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1130/2290] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:06 2022] PUT /identity/v3/projects/9696bd2b296b4aed865245217d79f2f6/users/5379b6cf2e004c16bc62171b98e7e7f0/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:06.647546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9135d-29f3-4907-901d-36348146f37a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.647863 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9135d-29f3-4907-901d-36348146f37a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.648685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9135d-29f3-4907-901d-36348146f37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.673085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d79ebf426b4ca8b5571815460ac2e7, user_id=0a1fe8eea9cb472784b71f7a37ca7b57, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:06.685653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd231b58-ffeb-4b14-b3e1-1297b6bfe2fa admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:06.698728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-80d9135d-29f3-4907-901d-36348146f37a None None] MFA Rules not processed for user `5379b6cf2e004c16bc62171b98e7e7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:06.708495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-80d9135d-29f3-4907-901d-36348146f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:06.714954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1161/2291] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:06 2022] PUT /identity/v3/projects/17d79ebf426b4ca8b5571815460ac2e7/users/0a1fe8eea9cb472784b71f7a37ca7b57/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:06.726326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:06.726966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:06.727280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:06.795384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] MFA Rules not processed for user `0a1fe8eea9cb472784b71f7a37ca7b57`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:06.797036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:06.844446 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1131/2292] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:06.860513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9135d-29f3-4907-901d-36348146f37a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.862291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-80d9135d-29f3-4907-901d-36348146f37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:06.884537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:06.888733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:06.946303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1162/2293] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:06.947113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:06.999785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9135d-29f3-4907-901d-36348146f37a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.003567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.004228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.004524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.004807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.022773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.024233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.048718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1132/2294] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:07.087117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f16fa4-358e-41d7-8d13-1f65a1560409 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.110938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.110938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.110938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.110938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.111361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.113565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.148930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1163/2295] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:06 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:07.151125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.162907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8d257-a200-45ae-854d-26b8f2a51c09 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.233882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.235068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.235371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.235662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.236498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.238826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.267937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1133/2296] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:07 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:07.269878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.293353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed4825-0824-4f78-bd7e-e44ff4cb2454 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.315709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.316859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.317489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.318231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.319736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.321549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.346332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1164/2297] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:07.351760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.362797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af9219-36fb-4a5f-a3ce-bcb672d473cf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.378671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.379607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.379897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.380216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] PATH_INFO: `/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e/users/2bd3216628bb480eab016dbfa0c14298/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.401366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8a1159ca1a4ee7aeb7f24a29804e3e, user_id=2bd3216628bb480eab016dbfa0c14298, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.405428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.433331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1134/2298] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:07 2022] PUT /identity/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e/users/2bd3216628bb480eab016dbfa0c14298/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:07.434199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.452928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24085c-1eeb-4710-9574-29f3f217de1d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.562929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.563672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.563965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.564248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.565034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.566769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5207656-7729-4771-8d9d-4ccb97ac460f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.581818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1165/2299] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:29:07.583021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.593688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aceeebe-00c5-4ec3-9e4b-9c26c0c78656 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.598864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.599813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.600277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.600550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] PATH_INFO: `/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e/users/2bd3216628bb480eab016dbfa0c14298/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.602600 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8a1159ca1a4ee7aeb7f24a29804e3e, user_id=2bd3216628bb480eab016dbfa0c14298, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.605249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.617119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1135/2300] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:07 2022] PUT /identity/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e/users/2bd3216628bb480eab016dbfa0c14298/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:07.628299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.628603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.628905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.659426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] MFA Rules not processed for user `2bd3216628bb480eab016dbfa0c14298`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:07.660914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:07.728361 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:07.753744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1166/2301] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:07.765063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.827552 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cf63a-c94d-411a-8208-69d4189d17b7 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.831200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.831847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.832133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.832408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.843624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.845253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.855738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1136/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:07.899069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:07.900502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:07.913023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670fab5-3d65-4f17-84e6-6276f80c9e5e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:07.916563 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:07.917244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:07.917530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:07.917806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:07.919566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:07.983019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:07.984691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6de74-6f63-470c-a12e-88bd9ed2eed8 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:07.999576 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1167/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:11.071850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:11.082121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:11.152100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba5972-544a-43ac-8dd8-9ce348996ac4 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:11.157558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:11.158546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:11.158904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:11.159374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] PATH_INFO: `/v3/users/523ca2a00b684a5faf9af1c405c7ac6d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:11.160597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523ca2a00b684a5faf9af1c405c7ac6d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:11.162888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:11.218781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523ca2a00b684a5faf9af1c405c7ac6d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:11.226722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523ca2a00b684a5faf9af1c405c7ac6d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:11.245543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523ca2a00b684a5faf9af1c405c7ac6d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:11.246410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523ca2a00b684a5faf9af1c405c7ac6d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:11.247111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523ca2a00b684a5faf9af1c405c7ac6d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:11.264769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1137/2304] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:11 2022] DELETE /identity/v3/users/523ca2a00b684a5faf9af1c405c7ac6d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:11.272503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:11.273596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:11.273946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:11.298041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:11.301080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:11.403541 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1168/2305] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:11.412174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:11.414084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:11.512057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efb971-35d6-40b3-a0f9-73c31c7b1128 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:11.517221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:11.518162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:11.518681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:11.519134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:11.540629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:11.542154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:11.555690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1138/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:12.045688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:12.053264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:12.080431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df1dd1-97d9-4d02-8ce9-c8260af5af18 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:12.096794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:12.100568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:12.102985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:12.103460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] PATH_INFO: `/v3/projects/5b307219f36a42afa6862612bd580f80` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:12.106027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b307219f36a42afa6862612bd580f80)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:12.112046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:12.174331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b307219f36a42afa6862612bd580f80', 'request_id': 'req-914ea388-525a-4152-8fd3-41e07b2a6a08'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.197394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b307219f36a42afa6862612bd580f80', 'request_id': 'req-914ea388-525a-4152-8fd3-41e07b2a6a08'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.211170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b307219f36a42afa6862612bd580f80', 'request_id': 'req-914ea388-525a-4152-8fd3-41e07b2a6a08'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.223990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] The token cache is being invalidate because project 5b307219f36a42afa6862612bd580f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:12.224290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.225162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.229516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1169/2307] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:12 2022] DELETE /identity/v3/projects/5b307219f36a42afa6862612bd580f80 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:12.230907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:12.244536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872169-b1ae-4ca5-a6bb-b666bd847481 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:12.280187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:12.316522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:12.317883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:12.319564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:12.320276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] PATH_INFO: `/v3/users/0a1fe8eea9cb472784b71f7a37ca7b57` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:12.322273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1fe8eea9cb472784b71f7a37ca7b57)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:12.324490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:12.390845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1fe8eea9cb472784b71f7a37ca7b57'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.402355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1fe8eea9cb472784b71f7a37ca7b57'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.422491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1fe8eea9cb472784b71f7a37ca7b57'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.425666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1fe8eea9cb472784b71f7a37ca7b57'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.426303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1fe8eea9cb472784b71f7a37ca7b57'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:12.440348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1139/2308] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:12 2022] DELETE /identity/v3/users/0a1fe8eea9cb472784b71f7a37ca7b57 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:13.196278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:13.197834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:13.210397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:13.220930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea388-525a-4152-8fd3-41e07b2a6a08 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:13.346850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:13.348299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:13.348676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:13.349090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] PATH_INFO: `/v3/projects/17d79ebf426b4ca8b5571815460ac2e7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:13.350782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d79ebf426b4ca8b5571815460ac2e7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:13.353648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:13.418273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d79ebf426b4ca8b5571815460ac2e7', 'request_id': 'req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.424488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d79ebf426b4ca8b5571815460ac2e7', 'request_id': 'req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.436184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d79ebf426b4ca8b5571815460ac2e7', 'request_id': 'req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.452243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] The token cache is being invalidate because project 17d79ebf426b4ca8b5571815460ac2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:13.452540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.453409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.457283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1170/2309] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:13 2022] DELETE /identity/v3/projects/17d79ebf426b4ca8b5571815460ac2e7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:13.462019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:13.464295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:13.478899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97446c-441c-4b24-b246-c4947c7389cc admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:13.549601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:13.550937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:13.552217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:13.552217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] PATH_INFO: `/v3/users/2bd3216628bb480eab016dbfa0c14298` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:13.553768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd3216628bb480eab016dbfa0c14298)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:13.555694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:13.621902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd3216628bb480eab016dbfa0c14298'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.626755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:13.631980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd3216628bb480eab016dbfa0c14298'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.649670 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd3216628bb480eab016dbfa0c14298'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.650409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd3216628bb480eab016dbfa0c14298'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.654973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c8edbce8-4cb0-40fb-89db-e93830e4c60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd3216628bb480eab016dbfa0c14298'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:13.667240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1140/2310] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:13 2022] DELETE /identity/v3/users/2bd3216628bb480eab016dbfa0c14298 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:29:14.243934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.245378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:14.259533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d1c34-1f5f-4fc7-bdd4-37ca48258bac admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.366642 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.367426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.367717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.367980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] PATH_INFO: `/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.371127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8a1159ca1a4ee7aeb7f24a29804e3e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.377378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.469968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:14.488412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8a1159ca1a4ee7aeb7f24a29804e3e', 'request_id': 'req-aa056d0f-02aa-458d-a60f-a66387f2512a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:14.499376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8a1159ca1a4ee7aeb7f24a29804e3e', 'request_id': 'req-aa056d0f-02aa-458d-a60f-a66387f2512a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:14.510409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8a1159ca1a4ee7aeb7f24a29804e3e', 'request_id': 'req-aa056d0f-02aa-458d-a60f-a66387f2512a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:14.526638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] The token cache is being invalidate because project 2b8a1159ca1a4ee7aeb7f24a29804e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:14.527011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:14.528501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-aa056d0f-02aa-458d-a60f-a66387f2512a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:14.530390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1171/2311] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:14 2022] DELETE /identity/v3/projects/2b8a1159ca1a4ee7aeb7f24a29804e3e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:29:14.536170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.536688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.536775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.559345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:14.561093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:14.650543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1141/2312] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:14.660912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.661217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.661507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.695777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:14.697164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:14.705922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1172/2313] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:14.710861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.721465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9be048-ae84-4900-a53a-9a062d5b544c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.726296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.726979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.727440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.727875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.729487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:14.729949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.731578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.749832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1142/2314] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:29:14.751116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.761299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcf274-d8ca-4628-94a2-837802daa581 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.766339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.767099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.767448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.767794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.772727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.774748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.805421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1173/2315] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:14.806623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.816502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4eb7b-8e6b-4483-98ff-a827a0e8e5bc admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.871380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.872047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.872337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.872619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.873361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.875277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.895365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1143/2316] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:14 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:29:14.900106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.910526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f01d32-a8ba-4776-bd18-7a950b145a8f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.915127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.915779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.916066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.916344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.917107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.918692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.930728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1174/2317] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:14.935515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.952217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb4e1-fc6c-4b7d-b3d3-682847d5bce9 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.956897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.957642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.957933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.958787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:14.959254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:14.960856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e969e-ead6-43a8-ae03-d1bf56001bbc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:14.971437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1144/2318] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:14.975760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:14.986431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94525ef5-903c-477f-9f16-301dac80711b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:14.992278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:14.993223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:14.993505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:14.993779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] PATH_INFO: `/v3/projects/eca85a6fa961411dae3a19c1c6582099/users/4bac1e75fe304680b0ab0ec2f34029f9/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.013373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca85a6fa961411dae3a19c1c6582099, user_id=4bac1e75fe304680b0ab0ec2f34029f9, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.016067 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.031000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1175/2319] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:14 2022] PUT /identity/v3/projects/eca85a6fa961411dae3a19c1c6582099/users/4bac1e75fe304680b0ab0ec2f34029f9/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:15.042204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.042585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.042771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.071236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] MFA Rules not processed for user `4bac1e75fe304680b0ab0ec2f34029f9`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:15.072444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:15.095318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:15.137131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1145/2320] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:15.141653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.203612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fb0c4-5b25-413b-a897-99ab38d6ca97 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.208319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.209079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.209381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.209732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.210542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.212488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.238544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1176/2321] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:15.243030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.253701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6562-542d-4334-a23b-95d900528cb9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.315689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.316374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.316659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.316995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.317697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.319788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.340519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1146/2322] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:15.344720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.355921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ec8e-1664-4c7d-a17c-78b509d17f27 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.361026 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.361694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.361978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.362252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.363199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.365597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.377149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1177/2323] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:15.381384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.392494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c42c8-5440-4a7f-9285-e94aaeae8486 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.398021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.398682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.398969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.399248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.399979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.401620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484800e5-5b4e-471d-8e42-d6c115e9d03e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.412395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1147/2324] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:15.416495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.426443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cab7a-6693-4c1b-8732-76f553ac0dc5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.431707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.432593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.432971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.433240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] PATH_INFO: `/v3/projects/9179e62d9cbb431b98615ade2edafeb4/users/7cc0115bfafb4e149a2c5180d022b4da/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.454811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9179e62d9cbb431b98615ade2edafeb4, user_id=7cc0115bfafb4e149a2c5180d022b4da, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.457527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.472759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1178/2325] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:15 2022] PUT /identity/v3/projects/9179e62d9cbb431b98615ade2edafeb4/users/7cc0115bfafb4e149a2c5180d022b4da/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:15.487677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.487978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.488270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.515914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] MFA Rules not processed for user `7cc0115bfafb4e149a2c5180d022b4da`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:15.517198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:15.586442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1148/2326] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:15.590976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.602187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:15.667942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738ed25-8573-4938-ae4f-b217ee663367 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.673679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.675207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.675747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.676294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.677335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.679365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.719301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1179/2327] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:29:15.723280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.735321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec1672-1eb2-4e42-b52b-4ca212809901 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.766342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:15.796279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.797054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.797414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.797843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.798561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.800521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.826424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1149/2328] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:15 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:15.827820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.839179 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079d9e-33ad-4e74-9e1d-64ae392607a8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.843936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.844646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.845010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.845429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.846245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.847868 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.860082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1180/2329] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:15.864338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.874369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235ce6e-e215-45e3-a63d-6fdd918ef483 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.878953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.879831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.880121 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.880426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] PATH_INFO: `/v3/projects/27d320ca0e154dc1a902640b5e151584/users/7fbc1c0c6b82490eafa55176324ef5ac/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.894797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d320ca0e154dc1a902640b5e151584, user_id=7fbc1c0c6b82490eafa55176324ef5ac, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.898417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.910988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1150/2330] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:15 2022] PUT /identity/v3/projects/27d320ca0e154dc1a902640b5e151584/users/7fbc1c0c6b82490eafa55176324ef5ac/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:15.915570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:15.925467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8ebd2-ff79-49cc-93cf-1e21aba4f30c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:15.980658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:15.981388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:15.981674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:15.981956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:15.982695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:15.984299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69e44f-3071-471e-bd77-ed1780ed2f17 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:15.995508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1181/2331] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:16.001843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:16.012410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed878801-ca69-47a7-aaee-7c5c7347f2bf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:16.017086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:16.017971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:16.018256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:16.018536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] PATH_INFO: `/v3/projects/27d320ca0e154dc1a902640b5e151584/users/7fbc1c0c6b82490eafa55176324ef5ac/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:16.020529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d320ca0e154dc1a902640b5e151584, user_id=7fbc1c0c6b82490eafa55176324ef5ac, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:16.023685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:16.034011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1151/2332] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:16 2022] PUT /identity/v3/projects/27d320ca0e154dc1a902640b5e151584/users/7fbc1c0c6b82490eafa55176324ef5ac/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:16.045076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:16.045383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:16.045660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:16.072331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] MFA Rules not processed for user `7fbc1c0c6b82490eafa55176324ef5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:16.073596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:16.141198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1182/2333] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:16.153475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:16.155178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:16.211386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105563-231c-4cf0-b7f1-94a2678aa70a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:16.214989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:16.215621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:16.215903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:16.216175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:16.270355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:16.272067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:16.281053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1152/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:18.905161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:18.906609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:18.915081 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:18.919676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d9e0f-8d6c-4622-a1a0-cbb0c3044d55 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:18.924467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:18.925172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:18.925461 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:18.925737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:18.992835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:18.994334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc79598-5680-4b48-a94b-bd7468dc786f service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:19.004609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1183/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:19.350085 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:19.353812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:19.411490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:19.438125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d2a3-f111-42d9-a287-7fe04116554f service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:19.442614 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:19.443589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:19.443855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:19.444135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] PATH_INFO: `/v3/users/5b84e1bb697b4c14a3d8133347524ccb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:19.445351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b84e1bb697b4c14a3d8133347524ccb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:19.447334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:19.501610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b84e1bb697b4c14a3d8133347524ccb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:19.514242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b84e1bb697b4c14a3d8133347524ccb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:19.529883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b84e1bb697b4c14a3d8133347524ccb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:19.530597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b84e1bb697b4c14a3d8133347524ccb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:19.531217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b84e1bb697b4c14a3d8133347524ccb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:19.550131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1153/2336] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:19 2022] DELETE /identity/v3/users/5b84e1bb697b4c14a3d8133347524ccb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:19.551876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:19.552583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:19.553364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:19.577021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:19.578431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:19.647215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:19.648693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:19.652125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1184/2337] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:19.662377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:19.663806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:19.732213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1a82-1ff3-4560-81d0-786b831beeec admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:19.734937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc33ac6-eb55-457c-bb6a-dca3510fa32e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:19.735910 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:19.736645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:19.736992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:19.737292 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:19.739285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:19.739409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:19.740569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:19.740863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:19.741180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:19.751112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:19.752551 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:19.762739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1185/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:19.807462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:19.808945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fe2aa-4035-41d8-aea7-26f0dddff140 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:19.819186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1154/2339] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:20.129881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.131416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.144399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34647db4-4d33-430e-9ca8-6b1ead784d6d service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.150038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.150786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.151073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.151349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] PATH_INFO: `/v3/projects/43eb84c5eba749dea293084a683fde29` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.152533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43eb84c5eba749dea293084a683fde29)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.154558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.195773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:20.202514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43eb84c5eba749dea293084a683fde29', 'request_id': 'req-754d424e-7706-49f0-9410-b56c6058295c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:20.208255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43eb84c5eba749dea293084a683fde29', 'request_id': 'req-754d424e-7706-49f0-9410-b56c6058295c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:20.216908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43eb84c5eba749dea293084a683fde29', 'request_id': 'req-754d424e-7706-49f0-9410-b56c6058295c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:20.229720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] The token cache is being invalidate because project 43eb84c5eba749dea293084a683fde29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:20.230015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:20.230656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-754d424e-7706-49f0-9410-b56c6058295c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:20.232184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1186/2340] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:20 2022] DELETE /identity/v3/projects/43eb84c5eba749dea293084a683fde29 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:20.237139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.237431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.237707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.255046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:20.256248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.260542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:20.322551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1155/2341] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:20.330173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.330476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.330774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.350952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:20.352132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.360581 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1187/2342] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:20.364543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.379087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d9a75-56ec-429f-9b10-7a6100650340 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.384083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.384599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.384900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.385191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.385926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:20.386238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.387827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.405051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1156/2343] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:29:20.406473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.416432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeaa8f-ab56-42b1-a3ab-8e2db79f75af None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.421236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.422549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.422549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.422549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.423332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.425319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.472852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1188/2344] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:20.475150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.486967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280643a-32b1-46a7-8393-24bae19ec955 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.565504 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.566172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.566619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.566760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.567621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.570806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.596214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1157/2345] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:29:20.599194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.609799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c62fbc-4d36-489f-bc15-02bbf7d53bed admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.615380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.616152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.616504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.616824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.617707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.619530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.630151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1189/2346] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:20.634579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.644668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061fbf-25b8-4d62-bab0-a8b9ea05e51f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.650922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.651608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.651893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.652178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.652945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.654550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501e0a9-f633-4301-b80f-21c263bfe4bd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.665343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1158/2347] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:20.669200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.680190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f37e7f-c5b8-43fc-986c-58ab4a7849b2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.685151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.686103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.686371 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.686649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] PATH_INFO: `/v3/projects/a3f8a9f5c50d41f1b376a3e377f3c57f/users/8795bcb01df446868c614e45f156eedc/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.702198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f8a9f5c50d41f1b376a3e377f3c57f, user_id=8795bcb01df446868c614e45f156eedc, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.705158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.720380 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1190/2348] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:20 2022] PUT /identity/v3/projects/a3f8a9f5c50d41f1b376a3e377f3c57f/users/8795bcb01df446868c614e45f156eedc/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:29:20.731909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.732268 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.732544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.761167 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] MFA Rules not processed for user `8795bcb01df446868c614e45f156eedc`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:20.762364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.831053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1159/2349] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:20.842651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:20.844401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:20.902436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a68f-1286-486a-ad5e-c602c11b7b9c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:20.905971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:20.906618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:20.906907 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:20.907188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:20.915708 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:20.917175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acabc21-fb1f-487c-92a8-2bf3fd8e45ff service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:20.926153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1191/2350] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:25.873449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:25.874992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:25.883657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:25.952912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e2d2c-1d6f-4d69-a159-b6b2bc2afc55 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:25.957437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:25.959648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:25.959940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:25.960225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] PATH_INFO: `/v3/users/4bac1e75fe304680b0ab0ec2f34029f9` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:25.962132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bac1e75fe304680b0ab0ec2f34029f9)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:25.964062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:26.031757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bac1e75fe304680b0ab0ec2f34029f9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.039855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bac1e75fe304680b0ab0ec2f34029f9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.052660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bac1e75fe304680b0ab0ec2f34029f9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.053394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bac1e75fe304680b0ab0ec2f34029f9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.054029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bac1e75fe304680b0ab0ec2f34029f9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.066137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1160/2351] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:25 2022] DELETE /identity/v3/users/4bac1e75fe304680b0ab0ec2f34029f9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:26.071884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f127cf-8daa-4663-842d-bae47158178c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:26.072350 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f127cf-8daa-4663-842d-bae47158178c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:26.072565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f127cf-8daa-4663-842d-bae47158178c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:26.092454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-43f127cf-8daa-4663-842d-bae47158178c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:26.093726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-43f127cf-8daa-4663-842d-bae47158178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:26.156000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1192/2352] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:26.166321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:26.168016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:26.225382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b010b2e-1073-4f51-ba4a-1db3cbca59f2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:26.228972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:26.229622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:26.229906 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:26.230182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:26.239004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:26.241527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:26.250820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1161/2353] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:29:26.658534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f127cf-8daa-4663-842d-bae47158178c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:26.659969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-43f127cf-8daa-4663-842d-bae47158178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:26.674522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f127cf-8daa-4663-842d-bae47158178c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:26.680091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:26.681342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:26.681627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:26.681913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] PATH_INFO: `/v3/projects/eca85a6fa961411dae3a19c1c6582099` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:26.683136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca85a6fa961411dae3a19c1c6582099)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:26.685066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:26.687587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:26.740086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca85a6fa961411dae3a19c1c6582099', 'request_id': 'req-41726e22-b6ac-4d66-9688-df260cd00d61'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.745493 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca85a6fa961411dae3a19c1c6582099', 'request_id': 'req-41726e22-b6ac-4d66-9688-df260cd00d61'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.753193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca85a6fa961411dae3a19c1c6582099', 'request_id': 'req-41726e22-b6ac-4d66-9688-df260cd00d61'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.764790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] The token cache is being invalidate because project eca85a6fa961411dae3a19c1c6582099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:26.765319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.765757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.767489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1193/2354] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:26 2022] DELETE /identity/v3/projects/eca85a6fa961411dae3a19c1c6582099 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:26.772622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:26.785391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae701ad-34b4-4ba8-a559-7988a1ca5980 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:26.810515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:26.844360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:26.845354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:26.845655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:26.845960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] PATH_INFO: `/v3/users/7cc0115bfafb4e149a2c5180d022b4da` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:26.847163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc0115bfafb4e149a2c5180d022b4da)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:26.849205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:26.910727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc0115bfafb4e149a2c5180d022b4da'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.920411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc0115bfafb4e149a2c5180d022b4da'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.934135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc0115bfafb4e149a2c5180d022b4da'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.934956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc0115bfafb4e149a2c5180d022b4da'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.935656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc0115bfafb4e149a2c5180d022b4da'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:26.949825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1162/2355] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:26 2022] DELETE /identity/v3/users/7cc0115bfafb4e149a2c5180d022b4da => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:29:27.232944 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:27.234470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:27.248464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41726e22-b6ac-4d66-9688-df260cd00d61 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:27.305889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:27.306696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:27.306977 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:27.307249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] PATH_INFO: `/v3/projects/9179e62d9cbb431b98615ade2edafeb4` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:27.308420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9179e62d9cbb431b98615ade2edafeb4)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:27.310389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:27.361521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9179e62d9cbb431b98615ade2edafeb4', 'request_id': 'req-0c9f8852-afd4-4ec7-9c57-770a370ba156'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.369574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9179e62d9cbb431b98615ade2edafeb4', 'request_id': 'req-0c9f8852-afd4-4ec7-9c57-770a370ba156'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.379926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9179e62d9cbb431b98615ade2edafeb4', 'request_id': 'req-0c9f8852-afd4-4ec7-9c57-770a370ba156'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.393150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] The token cache is being invalidate because project 9179e62d9cbb431b98615ade2edafeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:27.393444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.394104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.395687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1194/2356] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:27 2022] DELETE /identity/v3/projects/9179e62d9cbb431b98615ade2edafeb4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:27.400062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:27.401646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:27.414853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c85c85-bf72-4546-8dbb-e3afd477f2a8 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:27.472288 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:27.473183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:27.473467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:27.474271 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] PATH_INFO: `/v3/users/7fbc1c0c6b82490eafa55176324ef5ac` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:27.475465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fbc1c0c6b82490eafa55176324ef5ac)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:27.477425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:27.527178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fbc1c0c6b82490eafa55176324ef5ac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.528872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:27.535489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fbc1c0c6b82490eafa55176324ef5ac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.548095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fbc1c0c6b82490eafa55176324ef5ac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.548744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fbc1c0c6b82490eafa55176324ef5ac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.549382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b8e829a-10b9-4852-9701-c72953f13d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fbc1c0c6b82490eafa55176324ef5ac'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:27.564106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1163/2357] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:27 2022] DELETE /identity/v3/users/7fbc1c0c6b82490eafa55176324ef5ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:27.890524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:27.891974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:27.908904 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f8852-afd4-4ec7-9c57-770a370ba156 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:27.930007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:27.970929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:27.971803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:27.972090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:27.972395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] PATH_INFO: `/v3/projects/27d320ca0e154dc1a902640b5e151584` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:27.973573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d320ca0e154dc1a902640b5e151584)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:27.975520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.026832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d320ca0e154dc1a902640b5e151584', 'request_id': 'req-91d907dc-3095-4e62-b5c5-ee1a621d820c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:28.032143 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d320ca0e154dc1a902640b5e151584', 'request_id': 'req-91d907dc-3095-4e62-b5c5-ee1a621d820c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:28.040658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d320ca0e154dc1a902640b5e151584', 'request_id': 'req-91d907dc-3095-4e62-b5c5-ee1a621d820c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:28.051743 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] The token cache is being invalidate because project 27d320ca0e154dc1a902640b5e151584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:28.052324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:28.052657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-91d907dc-3095-4e62-b5c5-ee1a621d820c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:28.054222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1195/2358] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:27 2022] DELETE /identity/v3/projects/27d320ca0e154dc1a902640b5e151584 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:28.059260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.059711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.059895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.082025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:28.082025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:28.148298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1164/2359] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:28.156214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.156512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.156801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.176360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:28.178497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:28.187117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1196/2360] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:28.191210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.203953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07f95-5512-428f-984b-b16e20c36a9c None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.209880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.210562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.210872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.211146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.211888 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:29:28.212198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.213862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.231351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1165/2361] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:29:28.235495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.245857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c584c63-f96f-4a0e-9642-f7ba424b0140 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.250521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.251194 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.251484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.251766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.252464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.254404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.282544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1197/2362] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:29:28 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:28.286706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.297347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04881c5f-cda5-46a8-97b2-c07c16428345 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.354180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.354755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.355039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.355318 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.356020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.358001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.381820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1166/2363] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:29:28 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:29:28.385996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.396862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a70541-d5c5-4282-b29a-71b8b1a64e5d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.403057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.403736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.404000 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.404286 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.405085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.406683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.421516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1198/2364] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:29:28.422854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.435235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c5940-ff57-4b53-8578-a48cfbcc604d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.440461 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.441214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.441499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.442093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.443575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.445343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbbcf5-a664-4824-b67d-d6253f115071 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.456054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1167/2365] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:29:28.460165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.470200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fda4d-daac-4f2a-8472-70b87336f6ff admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.474983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.475856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.476141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.476417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] PATH_INFO: `/v3/projects/36d5b71389ac4e06a8c282c49f082ea6/users/a4a9fe79959a4fe4aac0807399845e5c/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.491692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d5b71389ac4e06a8c282c49f082ea6, user_id=a4a9fe79959a4fe4aac0807399845e5c, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.494802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.509538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1199/2366] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:29:28 2022] PUT /identity/v3/projects/36d5b71389ac4e06a8c282c49f082ea6/users/a4a9fe79959a4fe4aac0807399845e5c/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:28.520666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a8d87-f791-4d6f-af3e-4b87c48f3060 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.520996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a8d87-f791-4d6f-af3e-4b87c48f3060 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.521535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a8d87-f791-4d6f-af3e-4b87c48f3060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.547762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-437a8d87-f791-4d6f-af3e-4b87c48f3060 None None] MFA Rules not processed for user `a4a9fe79959a4fe4aac0807399845e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:28.549012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-437a8d87-f791-4d6f-af3e-4b87c48f3060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:28.569423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:28.619747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1168/2367] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:28.631819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:28.633548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:28.688594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:28.692559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab6fc-6b36-4055-9eb9-31c43d3c63c0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:28.697930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:28.698745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:28.699208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:28.699638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:28.708998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:28.710468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c0c4-d92f-4be2-bad3-a202416b97d5 service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:28.719491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1200/2368] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:36.207060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:36.207649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:36.208422 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:36.241820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] MFA Rules not processed for user `ac38b6c63c1649d68302de51cc3424c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:36.243453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:36.325803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1169/2369] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:38.715304 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:40.128986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693a92-ebd5-48b7-9760-0c9aa6792d8b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:40.129601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693a92-ebd5-48b7-9760-0c9aa6792d8b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:40.129601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693a92-ebd5-48b7-9760-0c9aa6792d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:40.157522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-97693a92-ebd5-48b7-9760-0c9aa6792d8b None None] MFA Rules not processed for user `b5f4125ecaec41e5ba5c4eaf00c43e07`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:40.158730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-97693a92-ebd5-48b7-9760-0c9aa6792d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.228802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1201/2370] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:40.393556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:40.395666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.440776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:40.441106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:40.441413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:40.465664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:40.467473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.572206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1202/2371] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:29:40.575853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340dea31-f99e-4bbd-ba50-5fd6e09daa42 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:40.590156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:40.591279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:40.591786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:40.592521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] PATH_INFO: `/v3/users/8795bcb01df446868c614e45f156eedc` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:40.594766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8795bcb01df446868c614e45f156eedc)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:40.597119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:40.605916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:40.635802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:40.637842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.681652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8795bcb01df446868c614e45f156eedc'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:40.707674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8795bcb01df446868c614e45f156eedc'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:40.750544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8795bcb01df446868c614e45f156eedc'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:40.751339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8795bcb01df446868c614e45f156eedc'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:40.751969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f8bbbb53-2463-4f97-8373-91c290222cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8795bcb01df446868c614e45f156eedc'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:40.774956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1170/2372] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:29:40 2022] DELETE /identity/v3/users/8795bcb01df446868c614e45f156eedc => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:40.781283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:40.781705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:40.782080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:40.823954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efc567-5c93-4a1b-a080-9549922bff5d None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:40.825036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:29:40.831812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.876481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:40.907402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1171/2373] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:29:40.917136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:40.917865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:40.918160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:40.918441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:40.918548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:40.919942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:40.920277 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:41.028226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:41.030210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:41.067800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1203/2374] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:29:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:29:41.091735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296481ea-7bfc-4b53-94d5-bab19cd41a8b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:41.098028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:41.098940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:41.099638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:41.100143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:41.128715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:41.132570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:41.139914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:29:41.151239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1172/2375] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:29:41.789198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:29:41.794168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:29:41.838213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a957-e9e6-4430-98f9-bbf440a74fa9 service swift] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:29:41.854998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:29:41.856941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:29:41.857263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:29:41.858792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] PATH_INFO: `/v3/projects/a3f8a9f5c50d41f1b376a3e377f3c57f` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:29:41.859770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f8a9f5c50d41f1b376a3e377f3c57f)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:29:41.862326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:29:41.935833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f8a9f5c50d41f1b376a3e377f3c57f', 'request_id': 'req-984ff32f-33e1-4cb5-bbaf-8050d793c933'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:41.943009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f8a9f5c50d41f1b376a3e377f3c57f', 'request_id': 'req-984ff32f-33e1-4cb5-bbaf-8050d793c933'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:41.963182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f8a9f5c50d41f1b376a3e377f3c57f', 'request_id': 'req-984ff32f-33e1-4cb5-bbaf-8050d793c933'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:41.980391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] The token cache is being invalidate because project a3f8a9f5c50d41f1b376a3e377f3c57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:29:41.980713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:41.981420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-984ff32f-33e1-4cb5-bbaf-8050d793c933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:29:41.983280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1204/2376] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:29:41 2022] DELETE /identity/v3/projects/a3f8a9f5c50d41f1b376a3e377f3c57f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:29:51.974147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:00.168491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:00.169983 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:00.183101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf856a2-4666-4874-8cc5-9420b82d18a6 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:00.246957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:00.247872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:00.248153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:00.248429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] PATH_INFO: `/v3/users/5379b6cf2e004c16bc62171b98e7e7f0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:00.249971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5379b6cf2e004c16bc62171b98e7e7f0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:00.252096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:00.311899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5379b6cf2e004c16bc62171b98e7e7f0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.321296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5379b6cf2e004c16bc62171b98e7e7f0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.334502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5379b6cf2e004c16bc62171b98e7e7f0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.335260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5379b6cf2e004c16bc62171b98e7e7f0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.335949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5379b6cf2e004c16bc62171b98e7e7f0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.349204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1173/2377] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:30:00 2022] DELETE /identity/v3/users/5379b6cf2e004c16bc62171b98e7e7f0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:00.353472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:00.353992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:00.354100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:00.372241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:00.373452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:00.432779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1205/2378] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:00.442447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:00.443841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:00.501193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eb529-1c21-4618-a139-1d4052298e4c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:00.505655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:00.506308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:00.506591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:00.507017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:00.516161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:00.517638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5040cf-45ea-456d-a548-dd5912c0b5ff service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:00.528139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1174/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:00.810510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:00.811986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:00.825137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ac775-3d96-4180-aaf4-e42b8b8bb661 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:00.829813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:00.830677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:00.830963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:00.831243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] PATH_INFO: `/v3/projects/9696bd2b296b4aed865245217d79f2f6` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:00.832475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9696bd2b296b4aed865245217d79f2f6)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:00.834445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:00.882099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9696bd2b296b4aed865245217d79f2f6', 'request_id': 'req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.887827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9696bd2b296b4aed865245217d79f2f6', 'request_id': 'req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.895711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9696bd2b296b4aed865245217d79f2f6', 'request_id': 'req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.907099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] The token cache is being invalidate because project 9696bd2b296b4aed865245217d79f2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:30:00.907389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.908031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-620284cf-df6b-4fdd-9f3a-7e17cb3dfc10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:00.909675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1206/2380] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:30:00 2022] DELETE /identity/v3/projects/9696bd2b296b4aed865245217d79f2f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:04.449103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:04.449588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:04.449952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:04.451394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1175/2381] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:30:04.459164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1a4b4-5040-4168-8f40-b4b2c74f298b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:04.461886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1a4b4-5040-4168-8f40-b4b2c74f298b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:04.462499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1a4b4-5040-4168-8f40-b4b2c74f298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:04.574022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d8b1a4b4-5040-4168-8f40-b4b2c74f298b None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:04.575811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d8b1a4b4-5040-4168-8f40-b4b2c74f298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:04.626381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:04.685217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1207/2382] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:30:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:04.698313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:04.700029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:04.734012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:04.772324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8915068-d2ad-456f-9dff-175fe6f639a3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:04.776000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:04.776658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:04.776990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:04.777270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:04.790780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:04.792246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfd788-38c5-4a38-b8c8-a73082d9b375 service glance] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:04.805937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1176/2383] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:04.878285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:04.878678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:04.879488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:04.881163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1208/2384] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:30:04.886189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e564c2-d878-4fca-8caf-1ed2f965fa52 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:04.886589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e564c2-d878-4fca-8caf-1ed2f965fa52 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:04.886943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e564c2-d878-4fca-8caf-1ed2f965fa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:04.907151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d5e564c2-d878-4fca-8caf-1ed2f965fa52 None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:04.908381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d5e564c2-d878-4fca-8caf-1ed2f965fa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:04.969911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1177/2385] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:30:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:04.984215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:04.986051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:05.002654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3927c-b124-44d3-9e33-db94b2cb98ef None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:05.007510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:05.008230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:05.008585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:05.008956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:05.017761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:05.020065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3f3e-7dab-4920-af15-dc88f1bc514b service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:05.030732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1209/2386] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:05.206176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:05.206492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:05.206776 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:05.209465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1178/2387] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:30:05.215034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:05.215420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:05.215812 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:05.248125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:05.249498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:05.260549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1210/2388] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:30:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:30:05.273805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:05.276480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:05.293042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefcbc3-7608-4fee-b8be-0c7526a73777 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:05.298626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:05.299376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:05.299747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:05.300090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:05.310751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:05.312174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753c5e6-5639-4be3-9dc4-b8c2f74fc276 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:05.322954 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1179/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:05.702970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:05.704998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:05.823960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718baed2-fe84-4cba-96d8-bd53b257a2ee None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:05.829908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:05.830731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:05.831094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:05.831435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:05.834556 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:05.854649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:05.856612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:05.868250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1211/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:10.975749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:10.976275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:10.976367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:10.997472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:10.998714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.060741 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1180/2391] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:30:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:11.076515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:11.078263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.146205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:11.148009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.153674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2aacbf-f5e1-4163-b83b-5ace4eff1611 service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:11.159624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:11.160714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.161100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.161357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.171874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:11.173814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3452a-5d41-420a-a02f-9771ec278ed0 service swift] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:11.179442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:11.187091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1212/2392] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:30:11.241989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3e450-9ac8-4843-94be-cb6b6bacebf7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:11.249883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:11.251109 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.251506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.251876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] PATH_INFO: `/v3/users/a4a9fe79959a4fe4aac0807399845e5c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.253227 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a9fe79959a4fe4aac0807399845e5c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:11.255800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:11.262518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:11.318235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a9fe79959a4fe4aac0807399845e5c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:11.328470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a9fe79959a4fe4aac0807399845e5c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:11.344406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a9fe79959a4fe4aac0807399845e5c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:11.345441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a9fe79959a4fe4aac0807399845e5c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:11.346080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-738261aa-468d-495f-bee2-323cd3c23fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a9fe79959a4fe4aac0807399845e5c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:11.372945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1181/2393] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:30:11 2022] DELETE /identity/v3/users/a4a9fe79959a4fe4aac0807399845e5c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:11.374764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.375066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.375414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.412731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:11.414156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.440073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.440476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.440763 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.466568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:11.467850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.527180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1213/2394] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:11.539996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:11.547261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1182/2395] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:30:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:30:11.549681 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.573622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:11.575663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:11.664166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4561e-08bc-4da6-952a-f322031ff027 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:11.668898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:11.669882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.670389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.670866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.675649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d5dea-5cad-4ba9-b319-e6e7d02d6440 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:11.685585 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:11.687752 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:11.688168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:11.689049 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:11.689832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:11.689971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c2d40-9406-4162-9f34-36f4276d6147 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:11.709106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:11.709106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:11.715525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1183/2396] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:30:11.717189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1214/2397] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:12.245742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:12.250608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:12.276203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2cfc4-0b90-442e-9726-922ba78e6a56 service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:12.285377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:12.286546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:12.286935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:12.287317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] PATH_INFO: `/v3/projects/36d5b71389ac4e06a8c282c49f082ea6` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:12.290399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d5b71389ac4e06a8c282c49f082ea6)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:12.297907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:12.328001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:12.405810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d5b71389ac4e06a8c282c49f082ea6', 'request_id': 'req-8471579b-ce82-4e64-8a93-92395154c915'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:12.415956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d5b71389ac4e06a8c282c49f082ea6', 'request_id': 'req-8471579b-ce82-4e64-8a93-92395154c915'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:12.423839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d5b71389ac4e06a8c282c49f082ea6', 'request_id': 'req-8471579b-ce82-4e64-8a93-92395154c915'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:12.446316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] The token cache is being invalidate because project 36d5b71389ac4e06a8c282c49f082ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:30:12.446757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:12.447283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8471579b-ce82-4e64-8a93-92395154c915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:12.448929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1184/2398] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:30:12 2022] DELETE /identity/v3/projects/36d5b71389ac4e06a8c282c49f082ea6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:12.457562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:12.457562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:12.458231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:12.533387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:12.538103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:12.673293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1215/2399] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:12.675013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:12.675873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:12.676329 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:12.701348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:12.702791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:12.721348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1185/2400] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:12.730093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:12.743233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324840a-8964-4429-be14-e669fa5e57fc None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:12.749319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:12.752025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:12.753449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:12.753813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:12.755012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:30:12.755392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:12.757547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:12.778767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1216/2401] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:30:12.780915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:12.794260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390604e-858c-4f10-979b-d29f2c35b1d7 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:12.806282 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:12.807240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:12.807754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:12.807883 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:12.808530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:12.812114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:12.857440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:12.858350 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1186/2402] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:30:12 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:30:12.876782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6070144-d5f9-4fac-878f-53e658534935 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:12.899157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:13.018041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:13.019605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.020405 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.021185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.022453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:13.026985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:13.079946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1217/2403] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:30:12 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:30:13.085343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:13.104870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8302288-dcce-411c-99ea-57fa26d1b6bf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:13.120950 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:13.123038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.124139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.126170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.128284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:13.134146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:13.161788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1187/2404] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:13.168259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:13.189223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48efdc8-91b3-4683-a75e-cb09c5ac6fb0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:13.198978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:13.199788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.200468 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.201445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.203710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:13.209127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e8773-e3f5-407e-83bc-34e20033fe2d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:13.232064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1218/2405] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:13.235349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:13.247367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219d416-714c-4f3c-8628-4c2f67a7b19c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:13.252838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:13.254385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.254843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.255276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] PATH_INFO: `/v3/projects/87ac732ab7df406c933081dd4939ae1e/users/a5d4beaae81b4b8b9e4057f012d18b8b/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.286201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ac732ab7df406c933081dd4939ae1e, user_id=a5d4beaae81b4b8b9e4057f012d18b8b, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:13.289120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:13.314402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1188/2406] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:30:13 2022] PUT /identity/v3/projects/87ac732ab7df406c933081dd4939ae1e/users/a5d4beaae81b4b8b9e4057f012d18b8b/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:13.320916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee08b4-f784-4f65-b9c4-9c407b2e193e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.320916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee08b4-f784-4f65-b9c4-9c407b2e193e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.321336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee08b4-f784-4f65-b9c4-9c407b2e193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.358475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-55ee08b4-f784-4f65-b9c4-9c407b2e193e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:13.360369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-55ee08b4-f784-4f65-b9c4-9c407b2e193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:13.525839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1219/2407] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:30:13.564538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:13.568133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:13.771816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479ae46-9b66-45d3-af51-5708baae1d10 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:13.775971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:13.776981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:13.777499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:13.777992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:13.795103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:13.808542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:13.839861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1189/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:22.126499 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:22.127078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:22.127078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:22.210721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] MFA Rules not processed for user `a5d4beaae81b4b8b9e4057f012d18b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:22.214475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:22.415301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1220/2409] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:22.416391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:22.435846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12299da8-f8c8-4125-935d-7feb6f16a33d service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:22.441596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:22.442376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:22.442714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:22.443191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:22.443846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:22.454156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:22.503562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1190/2410] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:30:22 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:30:22.506605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:22.559177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ff549-39c0-4ea7-aa72-07a185987145 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:22.589198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:22.705957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:22.707689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:22.708312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:22.708774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:22.713918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:22.719168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:22.771133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1221/2411] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:30:22 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:30:22.775034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:22.800091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9758ec-f746-4308-96d0-a07160e2f9b3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:22.824285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:22.825068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:22.825394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:22.825721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:22.826512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:22.829129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:22.847931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1191/2412] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:22.849049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:22.862435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72983c-740c-434a-9d53-d291ac95a6cd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:22.870530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:22.872036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:22.872862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:22.874190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] PATH_INFO: `/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a/users/3c8add597bca4c6a8ff7d088b46c98a9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:22.903597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fde697f7148a8bb9d07b563d9a28a, user_id=3c8add597bca4c6a8ff7d088b46c98a9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:22.914535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:22.984509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1222/2413] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:30:22 2022] PUT /identity/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a/users/3c8add597bca4c6a8ff7d088b46c98a9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:22.985807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:23.009984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e266a7a-5384-48c4-baba-16e9648f07b5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:23.024991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:23.148187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:23.148923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:23.149226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:23.149505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:23.150261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:23.151904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd458f05-a30c-4866-b3b8-38b4dc03e9b5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:23.189155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1192/2414] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:30:23.194236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:23.212976 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4e216-1ce9-49ea-94eb-8e61228175b1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:23.220190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:23.221199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:23.221491 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:23.221997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] PATH_INFO: `/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a/users/3c8add597bca4c6a8ff7d088b46c98a9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:23.227208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5fde697f7148a8bb9d07b563d9a28a, user_id=3c8add597bca4c6a8ff7d088b46c98a9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:23.229999 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6862b39-c9c6-461d-a014-5a88c429d955 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:23.258052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1223/2415] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:30:23 2022] PUT /identity/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a/users/3c8add597bca4c6a8ff7d088b46c98a9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:24.111853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:24.112436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:24.112788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:24.150028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] MFA Rules not processed for user `535a03ce19d346d59e6a6f6befb9dea0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:24.153310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:24.288917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1193/2416] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:32.403968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf03e-a03c-4b8f-9276-a14464e949e7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:32.404897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf03e-a03c-4b8f-9276-a14464e949e7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:32.405370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8cf03e-a03c-4b8f-9276-a14464e949e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:32.517423 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4e8cf03e-a03c-4b8f-9276-a14464e949e7 None None] MFA Rules not processed for user `3c8add597bca4c6a8ff7d088b46c98a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:32.526447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4e8cf03e-a03c-4b8f-9276-a14464e949e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:32.733024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1224/2417] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:32.741759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:32.743959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:32.953155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b93ff8-e2e3-4325-9dbc-fc3246408ff1 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:32.969365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:32.970372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:32.973164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:32.973703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:33.053359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:33.074711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:33.077358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:33.094378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1194/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:30:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:30:33.350401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1055-ab59-4a23-9f02-2623a502b4ee None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:33.350719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1055-ab59-4a23-9f02-2623a502b4ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:33.351107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1055-ab59-4a23-9f02-2623a502b4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:33.456398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-eb9c1055-ab59-4a23-9f02-2623a502b4ee None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:33.458150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-eb9c1055-ab59-4a23-9f02-2623a502b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:33.493562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:33.579253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1225/2419] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:30:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:33.591822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:33.593451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:33.793191 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8bdc3-eea0-406d-a389-e610b3b334d9 service glance] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:33.799098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:33.800558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:33.800925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:33.801310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:33.811263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:33.816447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:33.841544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1195/2420] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:30:35.662934 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:35.663473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:35.663883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:35.699873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:35.701550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:35.726878 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1226/2421] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:30:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:35.746661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:35.762694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84139bbe-f269-48a6-a61e-9f6348cf085a service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:35.770717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:35.771664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:35.771988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:35.772339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:35.782240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:35.786154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:35.809192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1196/2422] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:30:39.001847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:39.004008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:39.095487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48474ded-4c95-46fb-b052-8935fd910e73 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:39.100275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:39.101192 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:39.101663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:39.102180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] PATH_INFO: `/v3/users/ac38b6c63c1649d68302de51cc3424c1` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:39.103568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac38b6c63c1649d68302de51cc3424c1)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:39.105773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:39.177766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac38b6c63c1649d68302de51cc3424c1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.188733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac38b6c63c1649d68302de51cc3424c1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.196646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac38b6c63c1649d68302de51cc3424c1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.197396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac38b6c63c1649d68302de51cc3424c1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.212958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-33aeba8f-064d-40a3-bec5-42384b057bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac38b6c63c1649d68302de51cc3424c1'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.222766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1227/2423] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:30:39 2022] DELETE /identity/v3/users/ac38b6c63c1649d68302de51cc3424c1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:39.424058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:39.444539 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa199a3-f62c-4a01-84a5-f8d5569ae808 service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:39.571568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:39.573168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:39.573168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:39.573168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] PATH_INFO: `/v3/projects/c8055262b50d4e93b290315d0e57bd70` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:39.575478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8055262b50d4e93b290315d0e57bd70)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:39.581342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:39.654419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:39.659941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8055262b50d4e93b290315d0e57bd70', 'request_id': 'req-50d4b4c6-22ef-46ae-8b84-c12966f6a104'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.668819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8055262b50d4e93b290315d0e57bd70', 'request_id': 'req-50d4b4c6-22ef-46ae-8b84-c12966f6a104'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.676495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8055262b50d4e93b290315d0e57bd70', 'request_id': 'req-50d4b4c6-22ef-46ae-8b84-c12966f6a104'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.693233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] The token cache is being invalidate because project c8055262b50d4e93b290315d0e57bd70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:30:39.693629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.694536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-50d4b4c6-22ef-46ae-8b84-c12966f6a104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:30:39.700977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1197/2424] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:30:39 2022] DELETE /identity/v3/projects/c8055262b50d4e93b290315d0e57bd70 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:30:44.592439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:44.592939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:44.593059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:44.610500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] MFA Rules not processed for user `a5d4beaae81b4b8b9e4057f012d18b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:44.611689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:44.656517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:44.675120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1228/2425] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:30:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:45.387870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032153ca-fb46-4b93-a429-bba015ed3b80 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:45.388334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032153ca-fb46-4b93-a429-bba015ed3b80 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:45.388493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032153ca-fb46-4b93-a429-bba015ed3b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:45.408641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-032153ca-fb46-4b93-a429-bba015ed3b80 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:45.409996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-032153ca-fb46-4b93-a429-bba015ed3b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:45.485285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1198/2426] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:30:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:45.508910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:45.513807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:45.618860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5804e3-9a2b-4fd5-a99c-1c1f89892fa3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:45.625198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:45.626134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:45.626832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:45.627737 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:45.636700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:45.638336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:45.646922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1229/2427] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:30:48.034052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:48.034594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:48.034594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:48.054835 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:30:48.056056 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:30:48.063146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1199/2428] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:30:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:30:48.078873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:30:48.089312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf054c30-a0e1-48af-9c30-00852c348b58 service swift] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:30:48.094328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:30:48.094970 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:30:48.095261 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:30:48.096106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:30:48.104340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:30:48.105990 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42eb979-0414-4f2e-9479-295d5d2f8f1c service swift] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:30:48.115539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1230/2429] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:30:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:30:58.049069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=36,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:30:58.111072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=43 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:31:18.742041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:31:18.743601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:31:18.801078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d128f-c7ac-487f-aae7-056c4a7de718 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:31:18.804588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:31:18.805289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:31:18.805601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:31:18.805875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:31:18.807602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:31:18.869616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:31:18.871045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1026198a-8330-416c-8530-071b42cea5d4 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:31:18.880785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1200/2430] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:31:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:31:25.107447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae73e7-388e-4507-97d4-c1acef2ea9ff None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:31:25.107962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae73e7-388e-4507-97d4-c1acef2ea9ff None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:31:25.108066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae73e7-388e-4507-97d4-c1acef2ea9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:31:25.125962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-31ae73e7-388e-4507-97d4-c1acef2ea9ff None None] MFA Rules not processed for user `b5f4125ecaec41e5ba5c4eaf00c43e07`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:31:25.127161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-31ae73e7-388e-4507-97d4-c1acef2ea9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:31:25.181348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1231/2431] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:31:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:31:28.876525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=50 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:31:35.174806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=25 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:31:38.564560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:31:38.565077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:31:38.565181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:31:38.582454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] MFA Rules not processed for user `f1d7bb8f9d364c5198313fedd982478a`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:31:38.583664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:31:38.640718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1201/2432] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:31:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:31:46.743757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:31:46.744232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:31:46.744353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:31:46.762353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] MFA Rules not processed for user `535a03ce19d346d59e6a6f6befb9dea0`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:31:46.763550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:31:46.817890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1232/2433] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:31:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:31:48.633891 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=25 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:31:56.810365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=25 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:02.219072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:02.233087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc63be-575e-4e2e-ab86-0eaf8ece9d5d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:02.237380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:02.238044 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:02.238332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:02.238611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:02.240420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:02.310420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:02.311852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:02.323304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1202/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:03.510078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:03.511448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:03.524389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10f5f-ab13-4885-b15e-606319c5f4af None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:03.528038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:03.528706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:03.529029 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:03.529311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:03.532043 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:03.547060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:03.548823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:03.559808 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1233/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:12.318580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=34 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:13.554891 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:19.286456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:19.287815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:19.312861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9bcbd-5dec-4d10-8acd-03f7c9aff6b3 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:19.323543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:19.324205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:19.324605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:19.325480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:19.327833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:19.339037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:19.340640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:19.352130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1203/2436] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:29.347214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:31.244039 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:31.259331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dd2b-c523-4c29-a641-e0a933ff4bc2 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:31.265387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:31.267046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:31.267278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:31.267521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:31.269906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:31.284112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:31.286114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:31.299446 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1234/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:31.871382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:31.881237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2ec6-12fa-4e77-80dc-3e9b466fae6e service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:31.884846 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:31.885517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:31.885806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:31.886087 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:31.887941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:31.951530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:31.953023 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:31.962695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1204/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:35.153856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.155537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.232854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.234628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.247911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27c012-8048-4c71-b0eb-822b1bccb5c6 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.252708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.253629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.253912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.254188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] PATH_INFO: `/v3/users/b5f4125ecaec41e5ba5c4eaf00c43e07` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.255384 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f4125ecaec41e5ba5c4eaf00c43e07)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.256160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09632459-8540-43fc-afc8-4884d64e4341 service nova] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.257331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.261448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.262241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.262532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.262809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.265151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.278730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.280514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.294818 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1235/2439] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:35.313956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f4125ecaec41e5ba5c4eaf00c43e07'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.321596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f4125ecaec41e5ba5c4eaf00c43e07'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.336701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f4125ecaec41e5ba5c4eaf00c43e07'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.337407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f4125ecaec41e5ba5c4eaf00c43e07'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.338006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-93f1cdfc-c333-402f-8a4f-d5d8c60f9691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f4125ecaec41e5ba5c4eaf00c43e07'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.350100 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1205/2440] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:32:35 2022] DELETE /identity/v3/users/b5f4125ecaec41e5ba5c4eaf00c43e07 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:32:35.525764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.536727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1434cf-2a8e-4c1e-8950-b54d5bb7d458 service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.594893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.595790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.596075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.596353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] PATH_INFO: `/v3/projects/0e5f17f8ba8e4113a27a004ebb29917a` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.597547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5f17f8ba8e4113a27a004ebb29917a)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.599431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.645156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5f17f8ba8e4113a27a004ebb29917a', 'request_id': 'req-6204081d-fdd7-46eb-8658-c9b6c09af6cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.650485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5f17f8ba8e4113a27a004ebb29917a', 'request_id': 'req-6204081d-fdd7-46eb-8658-c9b6c09af6cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.657529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5f17f8ba8e4113a27a004ebb29917a', 'request_id': 'req-6204081d-fdd7-46eb-8658-c9b6c09af6cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.668245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] The token cache is being invalidate because project 0e5f17f8ba8e4113a27a004ebb29917a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:32:35.668558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.669201 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6204081d-fdd7-46eb-8658-c9b6c09af6cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:32:35.670719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1236/2441] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:32:35 2022] DELETE /identity/v3/projects/0e5f17f8ba8e4113a27a004ebb29917a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:32:35.676221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.676655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.676979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.694854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:35.696067 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.756505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1206/2442] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:35.764101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.764396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.764677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.777406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:35.784389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:35.786128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:35.793859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1237/2443] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:35.798478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.808379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811ab9-1710-44e5-9c16-4231de842892 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.813402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.814055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.814339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.814619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.815344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:32:35.815654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.817279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.818762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:35.833656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1207/2444] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:32:35.837935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.847810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d9ecb-c54a-46a7-9c73-3d5aa5e1c28c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.852459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.853147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.853439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.853719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.854421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.856365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.883216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1238/2445] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:32:35 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:32:35.887373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.897935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e889-7641-4926-9686-3a1d50716706 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.953646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.954317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.954605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.954886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.955586 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:35.957543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:35.976545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1208/2446] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:32:35 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:32:35.980640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:35.991763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daccd03-279d-4bbd-bad3-d9f59f27c41d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:35.996294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:35.996986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:35.997276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:35.997559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:35.998288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:36.001022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:36.011200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1239/2447] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:32:36.015068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:36.025673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114224-82b4-46d5-a19c-7dddd681af96 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:36.030148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:36.030795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.031193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.031474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.032223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:36.033829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c43e7-55e4-4e24-b6c5-ab1b6d8c1422 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:36.043745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1209/2448] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:32:36.048644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:36.058280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890908e2-94f9-43f4-89c3-73884d1806c0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:36.063489 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:36.064373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.064656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.064961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] PATH_INFO: `/v3/projects/d2cc2e1e741d4eeb849e07d9f2c371be/users/d5985474b0774d8ea0e62718de767310/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.079299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cc2e1e741d4eeb849e07d9f2c371be, user_id=d5985474b0774d8ea0e62718de767310, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:36.081974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:36.094918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1240/2449] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:32:36 2022] PUT /identity/v3/projects/d2cc2e1e741d4eeb849e07d9f2c371be/users/d5985474b0774d8ea0e62718de767310/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:32:36.100031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.100332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.100643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.118563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:36.119768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:36.180642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1210/2450] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:36.190100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:36.191849 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:36.251894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e98363-74e3-4ea5-9f91-714534b1739a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:36.255656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:36.256318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:36.256612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:36.256908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:36.270533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:36.272010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1639832-9b62-49d3-93bf-6ae7275a376f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:36.283112 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1241/2451] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:32:39.498294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:39.500407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:39.588814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9967a0-8e8f-478b-bef7-611b4c524999 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:39.600102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:39.600796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:39.601125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:39.601403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:39.604652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:39.616605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:39.684567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:39.686724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:39.712985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1211/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:40.283669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f259-1650-4186-8885-56cd2c1caaf3 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:40.285424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f259-1650-4186-8885-56cd2c1caaf3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:40.285810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f259-1650-4186-8885-56cd2c1caaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:40.319224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-1a50f259-1650-4186-8885-56cd2c1caaf3 None None] MFA Rules not processed for user `d5985474b0774d8ea0e62718de767310`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:40.321013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1a50f259-1650-4186-8885-56cd2c1caaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:40.405768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1242/2453] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:32:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:40.422330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:40.424265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:40.443143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab37d9-dbe3-4f98-bb9d-61edd05eb4de service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:40.449045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:40.450521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:40.450968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:40.451342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:40.464695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:40.466819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:40.480241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1212/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:42.072946 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:42.073487 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:42.073716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:42.091828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] MFA Rules not processed for user `d5985474b0774d8ea0e62718de767310`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:32:42.093099 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:42.100690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1243/2455] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:32:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:32:42.427850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:42.438842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da2dbf-5e98-4ce5-b0c9-98b3da02bc37 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:42.445428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:42.446266 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:42.446557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:42.446839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:42.448722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:42.461749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:42.463712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:42.473009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1213/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:42.821107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:42.823170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:42.867203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:32:42.924065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ccfb2-fe66-4efa-94b0-085bf732faf1 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:42.929215 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:42.929880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:42.930173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:42.930458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:42.932242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:42.944586 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:42.946332 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:42.957344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1244/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:45.314810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:45.323805 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e9012-1a34-4cf9-a07e-428250fb45e8 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:45.328003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:45.328793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:45.329170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:45.329434 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:45.331248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:45.341828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:45.343826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:45.352605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1214/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:51.606495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:51.619009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a4f88-d427-47be-bcf5-cafd2e6695ac service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:51.624434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:51.625144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:51.625433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:51.626198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:51.628829 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:51.642362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:51.644190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c8d87-3d11-47b6-98c2-95e2edcadbfd service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:51.655696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1245/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:32:54.526608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:32:54.545795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd893b6-5f31-44a0-ae99-f122323e6f80 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:32:54.553609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:32:54.554391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:32:54.554748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:32:54.555208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:32:54.557003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:32:54.570679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:32:54.572872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37feed90-e2e9-414e-8ee7-8f54900c9631 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:32:54.585129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1215/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:32:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:33:01.621718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:01.622695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:01.622980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:01.654683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:01.662942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:01.764519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1246/2461] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:33:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:01.771536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:01.772106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:01.772106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:01.811412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] MFA Rules not processed for user `535a03ce19d346d59e6a6f6befb9dea0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:01.813664 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:01.827644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:01.829770 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:01.961047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1216/2462] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:01.988084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b7b24-5385-4b97-88da-ba8185fff602 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:01.994875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:01.995542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:01.995972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:01.996263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:02.014041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:02.015637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58666295-c34d-4932-8f0e-cf5d79370712 service swift] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:02.026464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1247/2463] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:33:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:33:02.375307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:02.377814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:02.393506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71c2f7-06dc-4cc0-8263-fb80fae7331e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:02.397544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:02.398388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:02.398668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:02.399331 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:02.401538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:02.415168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:02.416962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:02.441401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1217/2464] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:03.111538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:03.112517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:03.113049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:03.136726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:03.138402 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:03.146998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1248/2465] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:33:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:03.163000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:03.181587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9511c56-9295-4b48-886b-008b08cc2b88 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:03.192273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:03.193092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:03.193477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:03.193766 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:03.205491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:03.208281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:03.222427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1218/2466] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:33:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:33:13.129594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=85 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:13.216345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=84 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:13.981158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:13.982855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:14.019685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4ed7d-e1e4-4dd1-b1d9-87d7e1e924f4 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:14.029733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:14.032015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:14.032452 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:14.032741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] PATH_INFO: `/v3/users/a5d4beaae81b4b8b9e4057f012d18b8b` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:14.034998 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d4beaae81b4b8b9e4057f012d18b8b)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:14.036965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:14.103593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d4beaae81b4b8b9e4057f012d18b8b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.112744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d4beaae81b4b8b9e4057f012d18b8b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.120567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d4beaae81b4b8b9e4057f012d18b8b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.121386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d4beaae81b4b8b9e4057f012d18b8b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.135835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d4beaae81b4b8b9e4057f012d18b8b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.144117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1249/2467] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:33:13 2022] DELETE /identity/v3/users/a5d4beaae81b4b8b9e4057f012d18b8b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:33:14.375981 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:14.391408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60182b1b-e806-4dcb-94cd-a670cfeff3f8 service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:14.452313 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:14.453760 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:14.454045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:14.454321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] PATH_INFO: `/v3/projects/87ac732ab7df406c933081dd4939ae1e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:14.455530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ac732ab7df406c933081dd4939ae1e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:14.457483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:14.512373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ac732ab7df406c933081dd4939ae1e', 'request_id': 'req-76d3c9c1-2b42-4638-be8e-5aa7150f73be'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.519685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ac732ab7df406c933081dd4939ae1e', 'request_id': 'req-76d3c9c1-2b42-4638-be8e-5aa7150f73be'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.527430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ac732ab7df406c933081dd4939ae1e', 'request_id': 'req-76d3c9c1-2b42-4638-be8e-5aa7150f73be'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.541786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] The token cache is being invalidate because project 87ac732ab7df406c933081dd4939ae1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:33:14.542073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.542725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.544602 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1219/2468] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:33:14 2022] DELETE /identity/v3/projects/87ac732ab7df406c933081dd4939ae1e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:33:14.548575 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:14.550024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:14.563120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2267f4e-72f3-401a-9a46-7839a67da1f7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:14.622210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:14.623397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:14.623852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:14.624141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] PATH_INFO: `/v3/users/3c8add597bca4c6a8ff7d088b46c98a9` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:14.625553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8add597bca4c6a8ff7d088b46c98a9)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:14.627485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:14.677513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8add597bca4c6a8ff7d088b46c98a9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.685052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8add597bca4c6a8ff7d088b46c98a9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.693236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8add597bca4c6a8ff7d088b46c98a9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.693915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8add597bca4c6a8ff7d088b46c98a9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.704673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:14.715226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-81a2262b-c37f-4772-b4d7-7443fa07f95a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8add597bca4c6a8ff7d088b46c98a9'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:14.723116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1250/2469] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:33:14 2022] DELETE /identity/v3/users/3c8add597bca4c6a8ff7d088b46c98a9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:33:14.899743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:14.901277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:14.915675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3c9c1-2b42-4638-be8e-5aa7150f73be admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:14.976204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:14.977106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:14.977395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:14.977675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] PATH_INFO: `/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:14.978859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5fde697f7148a8bb9d07b563d9a28a)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:14.980801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.016289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:15.029522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5fde697f7148a8bb9d07b563d9a28a', 'request_id': 'req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:15.035626 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5fde697f7148a8bb9d07b563d9a28a', 'request_id': 'req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:15.041356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5fde697f7148a8bb9d07b563d9a28a', 'request_id': 'req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:15.054221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] The token cache is being invalidate because project 9d5fde697f7148a8bb9d07b563d9a28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:33:15.054507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:15.055143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3b6eba33-0bf7-46b3-b311-59649f7fc0b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:33:15.056675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1220/2470] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:33:14 2022] DELETE /identity/v3/projects/9d5fde697f7148a8bb9d07b563d9a28a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:33:15.062648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.063082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.063360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.082671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:15.083869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:15.140838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1251/2471] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:15.148868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.149193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.149473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.169054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:15.170242 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:15.178081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1221/2472] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:15.182222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.191956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165eb91-38ca-47f6-b7f4-aee9c14aaaf9 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.196442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.197129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.197412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.197689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.198411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:33:15.198720 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.200349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.212147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1252/2473] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:33:15.216519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.226775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8551d4-00a1-40a7-9ae6-172d39539abc None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.231483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.232137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.232426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.232739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.233477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.235990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.261829 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1222/2474] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:33:15 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:33:15.265719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.276245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc95548-729e-4d12-a2e1-e6c51ddd365e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.333045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.333726 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.334016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.334295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.334993 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.336924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.356932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1253/2475] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:33:15 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:33:15.361929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.372171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d516cd9-92e8-4bb0-a1b5-f12784f87696 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.377487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.378250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.378598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.378986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.379792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.381478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.392997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1223/2476] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:33:15.396707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.407649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73550-27b4-4cd2-ae19-320b2db31491 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.412430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.413134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.413540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.413734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.414470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.416075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d63774-f9b1-4b99-90fe-521932da85c2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.428979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1254/2477] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:33:15.431152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.441976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23dae9-b537-4c0e-950f-aabec1349eaf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.446918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.447834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.448125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.448408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] PATH_INFO: `/v3/projects/193571b962dc4fa1aac98a6acbdc9370/users/728f4031df4148fe9ec9bf9ab31d25e5/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.463653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193571b962dc4fa1aac98a6acbdc9370, user_id=728f4031df4148fe9ec9bf9ab31d25e5, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.466351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.479991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1224/2478] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:33:15 2022] PUT /identity/v3/projects/193571b962dc4fa1aac98a6acbdc9370/users/728f4031df4148fe9ec9bf9ab31d25e5/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:33:15.485768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ac4-e8d9-4e78-bf29-c4a4ca1c9f2b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.486064 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ac4-e8d9-4e78-bf29-c4a4ca1c9f2b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.486347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9ac4-e8d9-4e78-bf29-c4a4ca1c9f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.503712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9a5a9ac4-e8d9-4e78-bf29-c4a4ca1c9f2b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:15.504945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9a5a9ac4-e8d9-4e78-bf29-c4a4ca1c9f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:15.554037 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:15.565126 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1255/2479] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:15.574663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:15.576387 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:15.634481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9f442-4801-4451-abf3-03eabe99e400 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:15.638137 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:15.638786 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:15.639090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:15.639392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:15.650467 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:15.652133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908b74e-e9d4-4a31-bc33-e04975a7993f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:15.661641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1225/2480] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:19.486221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:19.486765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:19.486765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:19.488472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1256/2481] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:33:19.495286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44234c0-37a5-4072-b78b-e4172454911c None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:19.495919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44234c0-37a5-4072-b78b-e4172454911c None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:19.496129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44234c0-37a5-4072-b78b-e4172454911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:19.530444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d44234c0-37a5-4072-b78b-e4172454911c None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:19.532068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d44234c0-37a5-4072-b78b-e4172454911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:19.634088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:19.664058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1226/2482] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:33:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:19.678616 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:19.680870 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:19.766839 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3add156-5cdb-435c-9489-3d34f05bdc67 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:19.775376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea2357b286df4e828b9f5024d71544ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:19.776073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:19.776367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:19.776730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:19.789693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:19.791154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d4e9b-dfeb-4f93-b07c-8cd60bb0302f service glance] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:19.820208 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1257/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:19.915089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153e03-a717-4672-b370-859f49c24643 None None] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:19.915386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153e03-a717-4672-b370-859f49c24643 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:19.915673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10153e03-a717-4672-b370-859f49c24643 None None] PATH_INFO: `/` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:19.917307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1227/2484] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:33:19.926013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c12e-1766-4e6b-9825-fc511131850d None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:19.926312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c12e-1766-4e6b-9825-fc511131850d None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:19.926587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9c12e-1766-4e6b-9825-fc511131850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:19.961303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-aff9c12e-1766-4e6b-9825-fc511131850d None None] MFA Rules not processed for user `e1d08f8e3b4d4376aff855f4bce68c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:19.964483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-aff9c12e-1766-4e6b-9825-fc511131850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:19.986604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1258/2485] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:33:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:20.004609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153e03-a717-4672-b370-859f49c24643 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:20.006445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-10153e03-a717-4672-b370-859f49c24643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:20.035899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10153e03-a717-4672-b370-859f49c24643 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:20.039991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:20.040661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:20.041001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:20.041261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:20.056335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:20.057974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21268254-6111-4a5d-a92b-78306669676f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:20.068646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1228/2486] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:20.177313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2cfa-7718-470d-b004-dd44dd222c93 None None] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:20.177766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2cfa-7718-470d-b004-dd44dd222c93 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:20.178238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2cfa-7718-470d-b004-dd44dd222c93 None None] PATH_INFO: `/` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:20.180421 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1259/2487] 10.222.0.23 () {54 vars in 921 bytes} [Thu Jun 16 00:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 00:33:20.186493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc446e29-69f3-4e76-8b26-8fd268434bf9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:20.187518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc446e29-69f3-4e76-8b26-8fd268434bf9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:20.187946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc446e29-69f3-4e76-8b26-8fd268434bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:20.209553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-dc446e29-69f3-4e76-8b26-8fd268434bf9 None None] MFA Rules not processed for user `d50db388db1446eeb60ccaa17ff55389`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:20.210956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-dc446e29-69f3-4e76-8b26-8fd268434bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:20.222012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1229/2488] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Jun 16 00:33:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:20.243078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2cfa-7718-470d-b004-dd44dd222c93 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:20.256756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e2cfa-7718-470d-b004-dd44dd222c93 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:20.261093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:20.262587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:20.262587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:20.262587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:20.276128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:20.279649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:20.294012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1260/2489] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:20.922420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:20.922955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:20.923117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:20.962373 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] MFA Rules not processed for user `728f4031df4148fe9ec9bf9ab31d25e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:20.964612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:21.094452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1230/2490] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:21.099071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.111131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c14b3-56c6-4e60-9be9-29a2da43cf65 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.117142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.117820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.118289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.118725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.119756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.121928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.168508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1261/2491] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:33:21 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:33:21.169502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.181934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc18dd-1b1a-48c3-ae1a-6f4f089baec8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.243332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.244028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.244321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.244608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.245352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.247294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.271649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1231/2492] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:33:21 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:33:21.273014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.284602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38019f-e387-4d89-bdfc-89fd45fe6656 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.289397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.290072 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.290358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.290644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.291484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.292988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.306877 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1262/2493] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:33:21.307998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.318985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326e606-e0c9-4e81-9399-6f3899e7f17f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.323894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.324793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.325119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.325441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] PATH_INFO: `/v3/projects/3255ace319f94590a46e9324d1f5be65/users/56630252f80749deaf6e5936a467cca9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.346789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3255ace319f94590a46e9324d1f5be65, user_id=56630252f80749deaf6e5936a467cca9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.351293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.366211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1232/2494] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:33:21 2022] PUT /identity/v3/projects/3255ace319f94590a46e9324d1f5be65/users/56630252f80749deaf6e5936a467cca9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:33:21.370764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.381517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4209580-0658-4521-a3cd-69e46e462670 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.450125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.450972 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.451381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.451698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.453478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.455260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006289e-7d04-4841-afcf-671c0981dc9f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.471613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1263/2495] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:33:21.475146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:21.488513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79f9bf-c878-4464-83e2-01735eb467f3 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:21.494306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:21.495218 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:21.495506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:21.495788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] PATH_INFO: `/v3/projects/3255ace319f94590a46e9324d1f5be65/users/56630252f80749deaf6e5936a467cca9/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:21.497837 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3255ace319f94590a46e9324d1f5be65, user_id=56630252f80749deaf6e5936a467cca9, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:21.501293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:21.517259 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1233/2496] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:33:21 2022] PUT /identity/v3/projects/3255ace319f94590a46e9324d1f5be65/users/56630252f80749deaf6e5936a467cca9/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:33:22.240500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c69c-fded-46eb-8cea-2137a25923aa None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:22.241034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c69c-fded-46eb-8cea-2137a25923aa None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:22.241611 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c69c-fded-46eb-8cea-2137a25923aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:22.266565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-71c5c69c-fded-46eb-8cea-2137a25923aa None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:22.268637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-71c5c69c-fded-46eb-8cea-2137a25923aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:22.346931 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:22.408848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1264/2497] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:33:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:22.423886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:22.427487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:22.445908 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:22.521993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada77f8-b3a5-460a-a139-1a038c9e6d02 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:22.529301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:22.530103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:22.531428 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:22.531724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:22.545068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:22.546858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:22.560152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1234/2498] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:33:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:33:28.266704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab1df-8da7-425e-8436-d590cc24d752 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:28.267826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab1df-8da7-425e-8436-d590cc24d752 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:28.268482 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ab1df-8da7-425e-8436-d590cc24d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:28.304531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d67ab1df-8da7-425e-8436-d590cc24d752 None None] MFA Rules not processed for user `56630252f80749deaf6e5936a467cca9`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:28.307115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d67ab1df-8da7-425e-8436-d590cc24d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:28.407174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1265/2499] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:28.421976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:28.423610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:28.544364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e7220-e912-4a41-9400-f14c1659398d service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:28.554162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:28.554873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:28.555163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:28.555449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:28.680448 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:28.682876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:28.697953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1235/2500] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:33:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:33:38.403305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=43 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:38.689595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=82 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:39.601155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1150-1f83-418e-9a9b-73d550922ac1 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:39.601646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1150-1f83-418e-9a9b-73d550922ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:39.601753 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1150-1f83-418e-9a9b-73d550922ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:39.621660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-336b1150-1f83-418e-9a9b-73d550922ac1 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:39.622869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-336b1150-1f83-418e-9a9b-73d550922ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:39.629911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1266/2501] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:33:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:39.642529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:33:39.654537 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed0aae-f2eb-488a-acce-1532d2e7e3e8 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:33:39.660151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:33:39.660826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:39.661153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:39.661431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:39.669629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:33:39.671793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:33:39.680485 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1236/2502] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:33:45.537074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:33:45.537636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:33:45.537636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:33:45.555782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] MFA Rules not processed for user `728f4031df4148fe9ec9bf9ab31d25e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:33:45.557031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:33:45.564022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1267/2503] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:33:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:33:49.676937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:33:55.550431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:02.765045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:02.766478 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:02.783077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431f71c-c923-4601-a82d-3679ae64930a service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:02.788523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:02.789203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:02.789492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:02.789775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:02.791522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:02.853317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:02.854964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:02.879946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1237/2504] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:02.913681 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:02.976958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eeca00-6cdf-45dd-9e78-2bfb55e6df2a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:02.983426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:02.984352 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:02.984578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:02.984855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:02.994811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:02.996531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:03.006437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1268/2505] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:12.871442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=28 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:13.002627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=26 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:21.561139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:21.571052 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc8246-22ba-4bb4-8c09-e17fbcbfc3cd service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:21.574689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:21.575397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:21.575745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:21.576107 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:21.579101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:21.646084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:21.647528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:21.659280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1238/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:25.153994 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.156766 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:25.176759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440aad2-ccfe-427c-a831-04b980151b16 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.181593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.182631 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.182992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.183384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] PATH_INFO: `/v3/users/535a03ce19d346d59e6a6f6befb9dea0` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.184662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535a03ce19d346d59e6a6f6befb9dea0)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.186910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.241625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535a03ce19d346d59e6a6f6befb9dea0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.249810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535a03ce19d346d59e6a6f6befb9dea0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.259161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535a03ce19d346d59e6a6f6befb9dea0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.259822 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535a03ce19d346d59e6a6f6befb9dea0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.273357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2619c438-0638-44fd-a526-fddcdfe544c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535a03ce19d346d59e6a6f6befb9dea0'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.279873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1269/2507] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:25 2022] DELETE /identity/v3/users/535a03ce19d346d59e6a6f6befb9dea0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:25.465566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.477193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358e6fa-20c9-41ee-b706-193f986207c5 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.537297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.538183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.538469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.538746 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] PATH_INFO: `/v3/projects/2d5d56c0c6444b23a1ac6cc2fcb0bb52` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.539943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5d56c0c6444b23a1ac6cc2fcb0bb52)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.541874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.586963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5d56c0c6444b23a1ac6cc2fcb0bb52', 'request_id': 'req-26ad4884-d7cf-4e8c-80df-692084150023'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.593590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5d56c0c6444b23a1ac6cc2fcb0bb52', 'request_id': 'req-26ad4884-d7cf-4e8c-80df-692084150023'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.599117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5d56c0c6444b23a1ac6cc2fcb0bb52', 'request_id': 'req-26ad4884-d7cf-4e8c-80df-692084150023'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.611355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] The token cache is being invalidate because project 2d5d56c0c6444b23a1ac6cc2fcb0bb52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:25.611650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.612294 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-26ad4884-d7cf-4e8c-80df-692084150023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:25.613865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1239/2508] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:25 2022] DELETE /identity/v3/projects/2d5d56c0c6444b23a1ac6cc2fcb0bb52 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:25.618757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.619053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.619336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.637106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:25.638319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:25.710053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1270/2509] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:25.718597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.718597 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.718982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.737442 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:25.738655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:25.746601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1240/2510] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:25.750802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.760497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53be0d-9b63-4bff-b533-ae282ce1263a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.765741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.766613 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.766902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.767184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.767920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:25.768235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.769953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.781919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1271/2511] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:25.786029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.795104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a8ac3f-c0ff-47a2-923c-7b3c786cb0d9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.799732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.800375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.800660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.800985 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.801682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.803565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.817538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:25.832257 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1241/2512] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:25 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:25.836515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.847065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4911f39-1512-41c5-a3ab-9325820c3637 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.902862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.903534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.903821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.904110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.904806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.906788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.913288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:25.928542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1272/2513] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:25 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:25.932797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.943673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116f614-ea03-4f8c-8641-c73919227f41 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.948196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.948854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.949178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.949451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.950189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.951778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.962568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1242/2514] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:25.966393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:25.976814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cebb52-e31f-453b-92a6-ab153d2f77b0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:25.981429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:25.982085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:25.982381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:25.982661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:25.983385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:25.985462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da90f7-87e4-4c80-962c-0c3021d4c6a8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:25.995184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1273/2515] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:25.999628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.010026 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb3f27-7c19-47b7-9a4e-a4b6678f9571 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.019645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.021530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.021821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.022556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] PATH_INFO: `/v3/projects/b4d426847c714c22981dbadf6914430b/users/4d7b827604b74a41b5f2d5b3a5bb8dde/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.039966 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d426847c714c22981dbadf6914430b, user_id=4d7b827604b74a41b5f2d5b3a5bb8dde, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.042647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.055355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1243/2516] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:25 2022] PUT /identity/v3/projects/b4d426847c714c22981dbadf6914430b/users/4d7b827604b74a41b5f2d5b3a5bb8dde/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:26.065776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.066103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.066394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.094291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] MFA Rules not processed for user `4d7b827604b74a41b5f2d5b3a5bb8dde`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:26.095580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:26.166278 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1274/2517] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:26.172755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.183175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.184707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:26.237608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50879de9-04b9-472c-9afc-10f03be1031f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.241869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18672035-f1f6-443d-8ed7-85c7bf94881e None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.243079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.243750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.244101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.244334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.245104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.245778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.246455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.246745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.247031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.247135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.248945 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:26.261519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.262963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.276163 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1244/2518] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:26 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:26.276363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1275/2519] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:26.277274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.340792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3438fa-ee7d-4e25-82a2-ce66ac8c331d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.346281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.347036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.347323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.347599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.348327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.350301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.372451 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1245/2520] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:26 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:26.376429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.388727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffcdaf-68d0-495e-a7de-04e9d595284f service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.394280 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.394949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.395239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.395520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.396254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.397889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.408750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1276/2521] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:26.412785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.423647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fceed5-c3ff-4347-8d55-c33f155637b9 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.428505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.429444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.429736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.430020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] PATH_INFO: `/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071/users/fd33ef2277c74b798554c361ea1efdc7/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.445779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b15f7d8c244c1c9a4597c2b5d95071, user_id=fd33ef2277c74b798554c361ea1efdc7, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.448456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.463104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1246/2522] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:26 2022] PUT /identity/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071/users/fd33ef2277c74b798554c361ea1efdc7/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:26.466925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.477930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c605b8c-9462-4864-908d-29b12330329a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.536059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.536662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.536989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.537275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.538021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.539666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99166e-d49c-4149-aa70-9b3455666712 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.541161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:26.553048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1277/2523] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:26.556054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.567284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e89b8-34e7-4147-9d08-4e23a3dde452 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.572009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.572933 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.573222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.573502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] PATH_INFO: `/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071/users/fd33ef2277c74b798554c361ea1efdc7/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.575518 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=79b15f7d8c244c1c9a4597c2b5d95071, user_id=fd33ef2277c74b798554c361ea1efdc7, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.580014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.585733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:26.592559 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1247/2524] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:26 2022] PUT /identity/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071/users/fd33ef2277c74b798554c361ea1efdc7/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:26.604020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.604391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.604791 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.633746 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] MFA Rules not processed for user `fd33ef2277c74b798554c361ea1efdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:26.634996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:26.714248 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1278/2525] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:26.727512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:26.729525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:26.818745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd81974-d2c2-420f-9139-05b351c121ba admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:26.822436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:26.823088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:26.823376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:26.823705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:26.833676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:26.835118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc015f-764e-4aef-af4b-571c8d3135b0 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:26.845182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1248/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:28.304344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:28.381311 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f5e7-aeaa-4e15-be13-695ba8acf96f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:28.385828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:28.386775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:28.387060 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:28.387333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] PATH_INFO: `/v3/users/4d7b827604b74a41b5f2d5b3a5bb8dde` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:28.388523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7b827604b74a41b5f2d5b3a5bb8dde)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:28.391509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:28.440366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7b827604b74a41b5f2d5b3a5bb8dde'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:28.448294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7b827604b74a41b5f2d5b3a5bb8dde'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:28.457975 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7b827604b74a41b5f2d5b3a5bb8dde'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:28.458778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7b827604b74a41b5f2d5b3a5bb8dde'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:28.471309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7b827604b74a41b5f2d5b3a5bb8dde'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:28.473564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:28.478677 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1279/2527] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:28 2022] DELETE /identity/v3/users/4d7b827604b74a41b5f2d5b3a5bb8dde => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:28.483713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:28.484864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:28.484864 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:28.501737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:28.502941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:28.570145 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1249/2528] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:28.580813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:28.582796 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:28.644022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386c04c-3c0e-4b9b-bdd6-8e4829625095 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:28.650243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:28.651550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:28.653947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:28.654589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:28.673319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:28.675938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:28.695803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1280/2529] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:29.208848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.210731 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.229210 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3806fd3-aee2-4332-864f-1c1fae5e2807 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.235377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.236976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.237592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.238365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] PATH_INFO: `/v3/projects/b4d426847c714c22981dbadf6914430b` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.239964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4d426847c714c22981dbadf6914430b)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.242806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.330086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4d426847c714c22981dbadf6914430b', 'request_id': 'req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.337925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4d426847c714c22981dbadf6914430b', 'request_id': 'req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.344969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4d426847c714c22981dbadf6914430b', 'request_id': 'req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.360006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] The token cache is being invalidate because project b4d426847c714c22981dbadf6914430b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:29.360306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.360991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.362841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1250/2530] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:29 2022] DELETE /identity/v3/projects/b4d426847c714c22981dbadf6914430b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:29.366666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.376652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0748ee2-86db-4d07-a2d9-31a3522c8aec service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.450857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.450857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.450857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.451339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] PATH_INFO: `/v3/users/fd33ef2277c74b798554c361ea1efdc7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.452349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd33ef2277c74b798554c361ea1efdc7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.454637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.494781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.497687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.507003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:29.556455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd33ef2277c74b798554c361ea1efdc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.569385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd33ef2277c74b798554c361ea1efdc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.578963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd33ef2277c74b798554c361ea1efdc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.583456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd33ef2277c74b798554c361ea1efdc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.597942 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd33ef2277c74b798554c361ea1efdc7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.604024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1281/2531] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:29 2022] DELETE /identity/v3/users/fd33ef2277c74b798554c361ea1efdc7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:29.604869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.608005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.616654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf23d-2a0c-47f4-a1da-f95c329ef70a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.641652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:29.691623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b829e24-87cf-4971-b24e-00ee33d3e2d3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:29.695538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.696217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.696510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.696787 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.698615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:29.699200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:29.700145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:29.700471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:29.700767 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] PATH_INFO: `/v3/users/f1d7bb8f9d364c5198313fedd982478a` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:29.702978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d7bb8f9d364c5198313fedd982478a)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.705178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.715988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:29.718260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:29.733541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1251/2532] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:29.760672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d7bb8f9d364c5198313fedd982478a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.770061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d7bb8f9d364c5198313fedd982478a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.778525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d7bb8f9d364c5198313fedd982478a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.779307 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d7bb8f9d364c5198313fedd982478a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.792543 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e1934c5a-3155-4f24-9bac-5a5191cdabce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d7bb8f9d364c5198313fedd982478a'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:29.799874 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1282/2533] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:29 2022] DELETE /identity/v3/users/f1d7bb8f9d364c5198313fedd982478a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:29.981911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:29.997582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b0dc0-f56f-40f2-a697-7a00e2333ab1 service placement] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.068822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.069720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.070001 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.070276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] PATH_INFO: `/v3/projects/cd8526627add42a597eff6fce7400885` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.071462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8526627add42a597eff6fce7400885)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.073394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.114053 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.122190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8526627add42a597eff6fce7400885', 'request_id': 'req-a137d04a-4af5-4262-b6c8-b98b73840881'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.129822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8526627add42a597eff6fce7400885', 'request_id': 'req-a137d04a-4af5-4262-b6c8-b98b73840881'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.135572 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8526627add42a597eff6fce7400885', 'request_id': 'req-a137d04a-4af5-4262-b6c8-b98b73840881'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.148977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] The token cache is being invalidate because project cd8526627add42a597eff6fce7400885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:30.149334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.150035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.151936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1252/2534] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:29 2022] DELETE /identity/v3/projects/cd8526627add42a597eff6fce7400885 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:30.157431 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873a8-5e52-489c-b98a-af9c4d19e531 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.157679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873a8-5e52-489c-b98a-af9c4d19e531 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.158044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0873a8-5e52-489c-b98a-af9c4d19e531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.166465 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.167931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.180178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-0a0873a8-5e52-489c-b98a-af9c4d19e531 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.181502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0a0873a8-5e52-489c-b98a-af9c4d19e531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.238221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137d04a-4af5-4262-b6c8-b98b73840881 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.242781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.243853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.244135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.244414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] PATH_INFO: `/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.245574 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] RBAC: Authorizing `identity:delete_project(project_id=79b15f7d8c244c1c9a4597c2b5d95071)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.247510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.250053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1283/2535] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.258247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191302-3001-4251-b6d4-b9782335231a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.258548 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191302-3001-4251-b6d4-b9782335231a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.258842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191302-3001-4251-b6d4-b9782335231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.277952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-78191302-3001-4251-b6d4-b9782335231a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.279258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-78191302-3001-4251-b6d4-b9782335231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.289129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1284/2536] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:30.293602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191302-3001-4251-b6d4-b9782335231a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.299778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79b15f7d8c244c1c9a4597c2b5d95071', 'request_id': 'req-0f2b136f-df94-4b7f-a61f-bdf1e036933a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.303807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191302-3001-4251-b6d4-b9782335231a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.306199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79b15f7d8c244c1c9a4597c2b5d95071', 'request_id': 'req-0f2b136f-df94-4b7f-a61f-bdf1e036933a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.312588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79b15f7d8c244c1c9a4597c2b5d95071', 'request_id': 'req-0f2b136f-df94-4b7f-a61f-bdf1e036933a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.325608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] The token cache is being invalidate because project 79b15f7d8c244c1c9a4597c2b5d95071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:30.325899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.326536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0f2b136f-df94-4b7f-a61f-bdf1e036933a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:30.328759 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1253/2537] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:30 2022] DELETE /identity/v3/projects/79b15f7d8c244c1c9a4597c2b5d95071 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:30.333976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe1df-f5e9-4efe-bf12-37087c0eba6f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.334263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe1df-f5e9-4efe-bf12-37087c0eba6f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.334541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe1df-f5e9-4efe-bf12-37087c0eba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.340025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.352667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7a9fe1df-f5e9-4efe-bf12-37087c0eba6f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.353911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7a9fe1df-f5e9-4efe-bf12-37087c0eba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.367225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.367900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.368195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.368470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.369234 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.369549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.371191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.386589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1285/2538] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:34:30.387528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.390479 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.402518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ede5-ed6c-4e7f-8ce3-7966a5c1a11e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.405154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.407484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.408124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.408409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.408689 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.409419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.411466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.417078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1254/2539] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:30.424298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.424661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.424980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.441141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1286/2540] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.442163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.444612 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.445850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.510534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb6771-aac0-4a7d-8b06-349afd1375b3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.512179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1255/2541] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:30.516626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.517044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.517334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.517459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.517612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.518322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.520242 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.526314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a173a8-d343-4d0f-8fc8-3dadc8f5705b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.531261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.531917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.532280 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.532527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.533357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:30.533825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.535527 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.541670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1287/2542] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.545732 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.547375 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1256/2543] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:34:30.551528 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.555625 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bb2a1-2fc7-4964-9084-8334f5a1394d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.560430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.561118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.561901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c200ed-fdc1-4f09-9ead-f4ac665abc30 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.562023 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.562199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.562936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.564539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.566778 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.567444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.567728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.568007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.568700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.570622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.577691 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1288/2544] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:30.579671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.589140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c335-fab2-400a-9b15-48d8aeee6c9d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.594073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.594736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.595021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.595299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.596021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.597756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.603886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1257/2545] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:30.609045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1289/2546] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:30.609215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.611542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.621120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfc7b3-9209-4895-a204-7fb84b702fbc admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.689454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf592a1-bbf4-48a5-8fd6-f46582196761 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.693757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.694769 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.694876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.695053 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.695326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] PATH_INFO: `/v3/projects/70cbc11c4bca4ed3a4edcd6cebe8fadf/users/2b638f188cb942e28930e8a145dc3e2c/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.695698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.696131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.696423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.697225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.699418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.717459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cbc11c4bca4ed3a4edcd6cebe8fadf, user_id=2b638f188cb942e28930e8a145dc3e2c, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.720747 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175735d-53b2-4c07-92e3-6f98baefd50b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.724867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1258/2547] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:30.729274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.741055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.741890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18404ef-6507-4227-bb7f-28af571d0caf admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.748470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1290/2548] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:30 2022] PUT /identity/v3/projects/70cbc11c4bca4ed3a4edcd6cebe8fadf/users/2b638f188cb942e28930e8a145dc3e2c/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:30.760349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.760967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.761320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.797116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] MFA Rules not processed for user `2b638f188cb942e28930e8a145dc3e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.798889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.808866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:30.821004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.821712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.821998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.822276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.823349 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.825017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.836079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1259/2549] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:30.840193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.850392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a587076-cd1a-4ff3-ba44-230bd9b0252c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.854909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.855564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.855852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.856131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.856855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.858486 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.867432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1291/2550] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:30.868330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1260/2551] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:30.871762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.872207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.881957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3fd09-b867-4e19-b0dc-2f4793dcaf0d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.882168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680316a0-da94-4b1a-b97c-990e043cecc7 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.886502 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.886638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.887239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.887396 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.887506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.887682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.887789 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.888061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] PATH_INFO: `/v3/projects/16c40d3987a14babb814692105e14d4e/users/bc187db459b84c6c98928bee03f6e92d/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.888467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.891378 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.904644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c40d3987a14babb814692105e14d4e, user_id=bc187db459b84c6c98928bee03f6e92d, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.908141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0fce5-5065-4cf8-9f21-84a6713483ff admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:30.923055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1292/2552] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:30.923260 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1261/2553] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:30 2022] PUT /identity/v3/projects/16c40d3987a14babb814692105e14d4e/users/bc187db459b84c6c98928bee03f6e92d/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:30.924301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:30.932402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.932773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.933150 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.959825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] MFA Rules not processed for user `bc187db459b84c6c98928bee03f6e92d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:30.961207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:30.987939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b41d9-9e50-41a3-ae83-1d240cd25d53 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:30.992702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:30.993410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:30.993698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:30.993982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:30.994674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:30.996592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.017608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1293/2554] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.021453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.029540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1262/2555] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:31.031665 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857f842-fc97-48b6-972f-001fbd2d505c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.034430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.037526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.038257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.039094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.039381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.040121 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.042276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.045151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e83c-b897-4939-88a2-f519d57ab6fb None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.049862 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.050522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.050807 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.051084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.051779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.053536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1294/2556] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:31.053722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.057653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.068155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056a6e8-46bc-4404-9ad8-78ba1f5a0f65 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.073048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.073933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.074375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.074670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] PATH_INFO: `/v3/projects/b88a74a675634bf9b936d38c5cda6cf8/users/2a9cfcdd0ddb49f8af6bd017d825adeb/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.081821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1263/2557] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:31.085581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.090255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88a74a675634bf9b936d38c5cda6cf8, user_id=2a9cfcdd0ddb49f8af6bd017d825adeb, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.092985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.106848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1295/2558] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/b88a74a675634bf9b936d38c5cda6cf8/users/2a9cfcdd0ddb49f8af6bd017d825adeb/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:31.110932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.135358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.151236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c2d37-3092-482b-8a78-195c80e5388f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.177801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.181690 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7e3ca-c1ee-4f77-8129-ff21c05e2e0e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.187749 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.188391 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.188672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.188969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.189728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.191324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.201141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1296/2559] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:31.205862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.214019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.214721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.215009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.215134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5cd42-8659-433d-bf40-b35e110d4428 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.215289 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.215989 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.218610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b3ae3-b13f-4b26-9040-1b2a8a66ccdc admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.219952 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.220852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.221169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.221451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] PATH_INFO: `/v3/projects/b88a74a675634bf9b936d38c5cda6cf8/users/2a9cfcdd0ddb49f8af6bd017d825adeb/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.223573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88a74a675634bf9b936d38c5cda6cf8, user_id=2a9cfcdd0ddb49f8af6bd017d825adeb, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.226324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.237995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1297/2560] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/b88a74a675634bf9b936d38c5cda6cf8/users/2a9cfcdd0ddb49f8af6bd017d825adeb/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:31.238791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1264/2561] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:31.242588 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.250947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.251241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.251524 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.278519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] MFA Rules not processed for user `2a9cfcdd0ddb49f8af6bd017d825adeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:31.279643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:31.307245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca04-6000-4812-b851-64fb7d54696a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.313252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.313974 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.314258 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.314536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.315301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.316925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.328073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1298/2562] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:31.331856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.342045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c115d7-2de1-4c40-bf95-51816ddba61f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.347270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.347917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.348204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.348488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.349264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.350306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1265/2563] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:31.351138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.361281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.362219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1299/2564] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:31.362625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:31.366157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.376087 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eb0b6-f2c2-4c8b-bbbb-c5e518908406 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.380734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.381644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.381932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.382214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] PATH_INFO: `/v3/projects/47b3ecb92c1048f1b12f635fcc964f5d/users/bcb09fd3ee354f6bb6ff04c959cea8b0/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.398477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b3ecb92c1048f1b12f635fcc964f5d, user_id=bcb09fd3ee354f6bb6ff04c959cea8b0, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.401149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01395520-b2cd-4ded-86a2-01de2a1a4c9d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.414466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1300/2565] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/47b3ecb92c1048f1b12f635fcc964f5d/users/bcb09fd3ee354f6bb6ff04c959cea8b0/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:31.421713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d57cf-3ede-4c8c-90dc-c9a8c51a59a5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.425141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.425440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.425722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.453018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] MFA Rules not processed for user `bcb09fd3ee354f6bb6ff04c959cea8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:31.454244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:31.476014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.476728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.477090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.477425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.491902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.514224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.526515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1301/2566] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:31.531264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.547533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.549058 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.558690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1266/2567] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:31.597498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d067d-6d19-4d3d-87bd-01dc6684bffb None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.602187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.602866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.603157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.603442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.604149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.606069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.635882 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.637451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1302/2568] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.641522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.644845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbabb0f0-afc0-40d3-a73c-9500aefa8c35 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.695656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.696403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.696692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.697003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.704082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49834c-75b5-4b79-b294-fc296441956a admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.709250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.710736 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.711024 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.711302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.712009 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.713967 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.737161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1303/2569] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:31.740983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.751236 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d529-0e96-4d47-b390-4224d88ff821 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.756086 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.756742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.757061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.757339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.758094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.759672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.760315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.761820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.770046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1304/2570] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:31.771176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1267/2571] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:31.773778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.783475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fcf25-e1d5-43c8-8af5-a77069241af6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.788097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.788992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.789283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.789560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] PATH_INFO: `/v3/projects/26f2828304604c5387f20474f26f7ab7/users/6d8c545c0c3e4add8bac946e1c51c568/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.804778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f2828304604c5387f20474f26f7ab7, user_id=6d8c545c0c3e4add8bac946e1c51c568, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.807434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.823080 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1305/2572] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/26f2828304604c5387f20474f26f7ab7/users/6d8c545c0c3e4add8bac946e1c51c568/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:31.827690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.838977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d932d43-7506-4342-9730-63783b7e7d9c service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.902102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.902911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.903192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.903495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.904225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.906389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56a5fb-759f-48d2-86b3-8aa165a20c50 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.911704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=100 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:31.921834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:31.922353 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1268/2573] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:31.932504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4207b5-1c29-4408-9bd3-b4709523b8a7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:31.938437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:31.939314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.939595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.939869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] PATH_INFO: `/v3/projects/26f2828304604c5387f20474f26f7ab7/users/6d8c545c0c3e4add8bac946e1c51c568/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.941842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f2828304604c5387f20474f26f7ab7, user_id=6d8c545c0c3e4add8bac946e1c51c568, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:31.944440 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:31.956335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1306/2574] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:31 2022] PUT /identity/v3/projects/26f2828304604c5387f20474f26f7ab7/users/6d8c545c0c3e4add8bac946e1c51c568/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:31.966174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:31.966468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:31.966751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:31.993855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] MFA Rules not processed for user `6d8c545c0c3e4add8bac946e1c51c568`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:31.995070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.063500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1269/2575] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:32.074200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.096580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:32.131118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c411708-ab8c-4f01-a517-c3d8e4308191 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.134723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.135369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.135655 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.135936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.145837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.147263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.157542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1307/2576] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:32.659309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:32.660740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:32.672973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46434951-9525-4df3-bc5a-f4e1133ec0a9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:32.676475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:32.677141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:32.677427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:32.677704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:32.733781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:32.735216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:32.744004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1270/2577] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:36.757418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:36.769484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ccffe-f261-43ea-a107-102c065ba3a1 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:36.774721 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:36.775363 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:36.775647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:36.775924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:36.836585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:36.838090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5aef8-4e9f-400a-b297-71772ef0ad87 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:36.847019 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1308/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:38.953638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.014659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d3659-a2b9-4c02-9c8c-476a196f179f service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.020019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.020899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.021211 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.021494 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] PATH_INFO: `/v3/users/bc187db459b84c6c98928bee03f6e92d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.022681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc187db459b84c6c98928bee03f6e92d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.024605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.052728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:39.072270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc187db459b84c6c98928bee03f6e92d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.080162 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc187db459b84c6c98928bee03f6e92d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.092065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc187db459b84c6c98928bee03f6e92d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.092690 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc187db459b84c6c98928bee03f6e92d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.094182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc187db459b84c6c98928bee03f6e92d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.106055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1271/2579] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:38 2022] DELETE /identity/v3/users/bc187db459b84c6c98928bee03f6e92d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:39.110727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.111203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.111306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.137938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:39.139148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.199254 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1309/2580] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:39.209520 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.210930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.268119 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31b43e-a7c0-4696-aab6-58de9e9def70 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.271711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.272357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.272650 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.272963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.282596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.284040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.293700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1272/2581] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:39.572740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.574308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:39.587274 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72699c-df96-4d1f-b995-8844f0a2f05c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.592151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.593157 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.593450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.593740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] PATH_INFO: `/v3/projects/16c40d3987a14babb814692105e14d4e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.594963 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c40d3987a14babb814692105e14d4e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.596905 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.648317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c40d3987a14babb814692105e14d4e', 'request_id': 'req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.653917 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:39.656432 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c40d3987a14babb814692105e14d4e', 'request_id': 'req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.666115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c40d3987a14babb814692105e14d4e', 'request_id': 'req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.678718 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] The token cache is being invalidate because project 16c40d3987a14babb814692105e14d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:39.679035 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.679687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.681372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1310/2582] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:39 2022] DELETE /identity/v3/projects/16c40d3987a14babb814692105e14d4e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:39.685601 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:39.695140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ca7a4-e48b-4e2a-a101-29790e9338d7 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:39.763203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:39.764093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:39.764382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:39.764662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] PATH_INFO: `/v3/users/bcb09fd3ee354f6bb6ff04c959cea8b0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:39.767544 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb09fd3ee354f6bb6ff04c959cea8b0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:39.769515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:39.831196 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb09fd3ee354f6bb6ff04c959cea8b0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.832924 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:39.839965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb09fd3ee354f6bb6ff04c959cea8b0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.855408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb09fd3ee354f6bb6ff04c959cea8b0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.856237 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb09fd3ee354f6bb6ff04c959cea8b0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.856973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb09fd3ee354f6bb6ff04c959cea8b0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:39.870726 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1273/2583] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:39 2022] DELETE /identity/v3/users/bcb09fd3ee354f6bb6ff04c959cea8b0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:40.177692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.179202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.192638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c282-ccc4-45b6-83c7-0bb2a167f2cb admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:40.251516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:40.252439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.252722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.253036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] PATH_INFO: `/v3/projects/47b3ecb92c1048f1b12f635fcc964f5d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.254283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b3ecb92c1048f1b12f635fcc964f5d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:40.256987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.305292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b3ecb92c1048f1b12f635fcc964f5d', 'request_id': 'req-c8ca1454-9275-458d-86e7-6572c7d09303'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.310951 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b3ecb92c1048f1b12f635fcc964f5d', 'request_id': 'req-c8ca1454-9275-458d-86e7-6572c7d09303'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.318292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b3ecb92c1048f1b12f635fcc964f5d', 'request_id': 'req-c8ca1454-9275-458d-86e7-6572c7d09303'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.329185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] The token cache is being invalidate because project 47b3ecb92c1048f1b12f635fcc964f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:40.329475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.330095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.331618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1311/2584] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:40 2022] DELETE /identity/v3/projects/47b3ecb92c1048f1b12f635fcc964f5d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:40.335512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.337180 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.350411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14bfa2-d3b1-4221-9712-3323076a00d9 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:40.410197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:40.412337 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.412629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.412945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] PATH_INFO: `/v3/users/6d8c545c0c3e4add8bac946e1c51c568` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.414440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8c545c0c3e4add8bac946e1c51c568)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:40.416481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.466175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8c545c0c3e4add8bac946e1c51c568'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.474263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8c545c0c3e4add8bac946e1c51c568'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.486246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8c545c0c3e4add8bac946e1c51c568'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.486904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8c545c0c3e4add8bac946e1c51c568'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.487501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-76b5c0e3-1a0f-4665-98fd-9f40e3f444c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8c545c0c3e4add8bac946e1c51c568'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.499279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1274/2585] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:40 2022] DELETE /identity/v3/users/6d8c545c0c3e4add8bac946e1c51c568 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:40.775104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:40.776559 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.790285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca1454-9275-458d-86e7-6572c7d09303 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:40.849390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:40.850256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.850544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.850823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] PATH_INFO: `/v3/projects/26f2828304604c5387f20474f26f7ab7` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.852004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f2828304604c5387f20474f26f7ab7)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:40.853965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:40.870574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:40.904152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f2828304604c5387f20474f26f7ab7', 'request_id': 'req-a090db01-b375-4a22-bfa2-f78e7d76a389'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.909930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f2828304604c5387f20474f26f7ab7', 'request_id': 'req-a090db01-b375-4a22-bfa2-f78e7d76a389'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.917205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f2828304604c5387f20474f26f7ab7', 'request_id': 'req-a090db01-b375-4a22-bfa2-f78e7d76a389'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.928336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] The token cache is being invalidate because project 26f2828304604c5387f20474f26f7ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:40.928630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.929276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-a090db01-b375-4a22-bfa2-f78e7d76a389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:40.930948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1312/2586] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:40 2022] DELETE /identity/v3/projects/26f2828304604c5387f20474f26f7ab7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:40.937447 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:40.937753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:40.938034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:40.955712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:40.956941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:40.995639 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:41.016443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1275/2587] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:41.026002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.026291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.026568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.050894 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:41.053872 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:41.063313 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1313/2588] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:41.067818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.081051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a597835-5b0f-4a4d-82d3-ce38875b919a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.085696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.086364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.086648 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.086927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.087695 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:41.088032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.089663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.102997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1276/2589] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:41.107481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.117463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906bfea-4065-4e00-a732-df4ab9d5e256 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.122176 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.122833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.123118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.123396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.124097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.127047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.157816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1314/2590] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:41.158832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.168947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25d24e-90ec-4cda-8343-70837f63ce1f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.238361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.239476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.240000 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.240338 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.241654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.246445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.272013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1277/2591] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:41.276239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.287570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964a9f8-a54a-412a-986a-2e2f9d1870f1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.292308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.293014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.293309 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.293594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.294494 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.296656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.307811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1315/2592] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:41.313849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.325519 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961d057-3a80-4bb0-a8ad-09d0770ef128 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.331276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.331959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.332341 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.332526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.333765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.336009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d882b-a224-49b8-9643-5c054274d863 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.352958 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1278/2593] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:41.353529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.369585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556a7-4030-4deb-9267-ad1de5063cf9 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.380324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.381386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.381663 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.381961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] PATH_INFO: `/v3/projects/8a6fd51450ef47829b7396f68a285248/users/7fa2f663518046ec9b5d58b76bb88fae/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.406562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6fd51450ef47829b7396f68a285248, user_id=7fa2f663518046ec9b5d58b76bb88fae, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.409444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.426609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1316/2594] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:41 2022] PUT /identity/v3/projects/8a6fd51450ef47829b7396f68a285248/users/7fa2f663518046ec9b5d58b76bb88fae/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:41.438230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.438523 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.438804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.475556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] MFA Rules not processed for user `7fa2f663518046ec9b5d58b76bb88fae`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:41.485783 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:41.556034 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1279/2595] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:41.560496 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.620442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f859ef-00fc-46ec-bd53-202352a7323e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.625105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.625762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.626047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.626324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.627010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.628911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.657145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1317/2596] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:41.661135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.672336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5930abf-4262-4687-af08-8caa2d5b01d3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.710204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:41.731114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.731858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.732151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.732438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.733174 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.735791 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.757660 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1280/2597] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:41.758794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.769362 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed2f0-c824-4a91-bfa5-d071b6822bc8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.773948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.774668 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.774960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.775245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.775982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.777626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.788701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1318/2598] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:41.792673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.803452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037edfc-a577-4d06-a698-91f3d994165c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.808083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.808996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.809286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.809571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] PATH_INFO: `/v3/projects/01c425e8edab46fe90100937927ab8e3/users/6eaf3aec195e42d49e07f00dab006e94/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.824155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c425e8edab46fe90100937927ab8e3, user_id=6eaf3aec195e42d49e07f00dab006e94, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.826861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.840037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1281/2599] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:41 2022] PUT /identity/v3/projects/01c425e8edab46fe90100937927ab8e3/users/6eaf3aec195e42d49e07f00dab006e94/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:41.844475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.854529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc94a8d-49c2-419e-b39e-811a89f92fd6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.866693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:41.913010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.913603 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.913887 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.914164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.914888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.916466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddb3e7-c1ef-4368-9835-22b8cb62aa14 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.927337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1319/2600] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:41.931555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:41.942512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a33df0-00e2-4585-ae0d-c9060e17e03a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:41.947065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:41.947940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.948222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.948496 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] PATH_INFO: `/v3/projects/01c425e8edab46fe90100937927ab8e3/users/6eaf3aec195e42d49e07f00dab006e94/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:41.950470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01c425e8edab46fe90100937927ab8e3, user_id=6eaf3aec195e42d49e07f00dab006e94, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:41.953615 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:41.963907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1282/2601] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:41 2022] PUT /identity/v3/projects/01c425e8edab46fe90100937927ab8e3/users/6eaf3aec195e42d49e07f00dab006e94/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:41.975664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:41.975959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:41.976266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.002260 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] MFA Rules not processed for user `6eaf3aec195e42d49e07f00dab006e94`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:42.003458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:42.068889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1320/2602] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:42.080409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.082106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:42.138555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef154-3ce6-4bd8-bd38-b39f62b5d394 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.143458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.144103 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.144388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.144669 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.153567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.155003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd311d-53c5-40f2-ab46-97ae5e42718c service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.165462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1283/2603] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:42.711535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.773189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b684f81-3106-47e4-ae22-9a669afbb929 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:42.777656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:42.778530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.778826 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.779108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] PATH_INFO: `/v3/users/7fa2f663518046ec9b5d58b76bb88fae` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.780290 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa2f663518046ec9b5d58b76bb88fae)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:42.782292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:42.826300 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:42.836814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa2f663518046ec9b5d58b76bb88fae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:42.845817 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa2f663518046ec9b5d58b76bb88fae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:42.853119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa2f663518046ec9b5d58b76bb88fae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:42.853768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa2f663518046ec9b5d58b76bb88fae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:42.865920 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa2f663518046ec9b5d58b76bb88fae'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:42.872109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1321/2604] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:42 2022] DELETE /identity/v3/users/7fa2f663518046ec9b5d58b76bb88fae => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:42.877132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:42.877607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:42.877701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:42.896002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:42.897232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:42.963762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1284/2605] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:42.974862 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:42.976530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:43.038693 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7202-37d6-40c6-bc87-631368b9d41f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.042982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.043633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.043922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.044198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.056229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.057700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:43.068730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1322/2606] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:43.382623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.384092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:43.399013 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6335f98-3301-4864-989a-5e4a6db88e09 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.407071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.409562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.410898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.411250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] PATH_INFO: `/v3/projects/8a6fd51450ef47829b7396f68a285248` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.413437 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6fd51450ef47829b7396f68a285248)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.415419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:43.456409 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:43.476339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6fd51450ef47829b7396f68a285248', 'request_id': 'req-6530ce73-79da-4cd1-be34-85760a31cb3c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.482682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6fd51450ef47829b7396f68a285248', 'request_id': 'req-6530ce73-79da-4cd1-be34-85760a31cb3c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.489684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6fd51450ef47829b7396f68a285248', 'request_id': 'req-6530ce73-79da-4cd1-be34-85760a31cb3c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.505575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] The token cache is being invalidate because project 8a6fd51450ef47829b7396f68a285248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:43.505904 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.506548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.509298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1285/2607] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:43 2022] DELETE /identity/v3/projects/8a6fd51450ef47829b7396f68a285248 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:43.513368 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.525078 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47eac0-2202-48ef-b784-558a021b93c3 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:43.587587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:43.588585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:43.589147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:43.589651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] PATH_INFO: `/v3/users/6eaf3aec195e42d49e07f00dab006e94` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:43.591182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eaf3aec195e42d49e07f00dab006e94)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:43.594048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:43.648523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eaf3aec195e42d49e07f00dab006e94'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.665070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eaf3aec195e42d49e07f00dab006e94'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.674938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:43.678100 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eaf3aec195e42d49e07f00dab006e94'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.679269 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eaf3aec195e42d49e07f00dab006e94'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.697129 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbd8a963-8d8b-4324-baf2-2a2068dd5c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eaf3aec195e42d49e07f00dab006e94'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:43.704435 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1323/2608] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:43 2022] DELETE /identity/v3/users/6eaf3aec195e42d49e07f00dab006e94 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:43.990514 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:43.992016 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.010261 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6530ce73-79da-4cd1-be34-85760a31cb3c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.073859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.074750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.075036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.075334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] PATH_INFO: `/v3/projects/01c425e8edab46fe90100937927ab8e3` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.076493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=01c425e8edab46fe90100937927ab8e3)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.078977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.122735 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01c425e8edab46fe90100937927ab8e3', 'request_id': 'req-5859976b-abfa-418e-a9ed-4e6436d31c1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.129273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01c425e8edab46fe90100937927ab8e3', 'request_id': 'req-5859976b-abfa-418e-a9ed-4e6436d31c1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.134629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01c425e8edab46fe90100937927ab8e3', 'request_id': 'req-5859976b-abfa-418e-a9ed-4e6436d31c1c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.147182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] The token cache is being invalidate because project 01c425e8edab46fe90100937927ab8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:44.147473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.148113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5859976b-abfa-418e-a9ed-4e6436d31c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:44.149688 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1286/2609] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:43 2022] DELETE /identity/v3/projects/01c425e8edab46fe90100937927ab8e3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:44.154522 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.154995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.155090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.172937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:44.174149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.231014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1324/2610] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:44.239314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.239546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.239830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.258188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:44.259385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.267587 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1287/2611] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:44.271601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.281169 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce428cf7-3b9f-4986-a8dd-4a1db1bc2caf None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.285706 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.286354 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.286643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.286925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.287653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:44.287968 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.289607 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.301815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1325/2612] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:34:44.305941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.316278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76cb2-5213-41b8-a70b-d8aa567e4540 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.320949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.321603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.321894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.322178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.322872 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.324765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.349751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1288/2613] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:44.353804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.364397 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe2f0f-2a28-43d1-8229-1988726e9db2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.419713 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.420389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.420680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.421526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.422225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.424142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.446059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1326/2614] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:44.450037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.462870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981164de-f3a1-42e2-943c-5e742384af73 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.467553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.468371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.468744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.469065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.469810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.471420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.482388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1289/2615] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:44.487003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.497529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540b400-92a2-4187-b6ca-2f8f99e14884 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.502922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.503626 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.503914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.504196 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.504965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.506561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa5a47-9060-4582-92c7-83d6692662d3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.517013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1327/2616] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:44.521110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.531658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941da297-c3ef-4d1f-9535-c08b23b9e298 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.536355 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.537274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.537570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.537847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] PATH_INFO: `/v3/projects/8d633a90bcd14e4da76366cd8812c97d/users/0d91a59becd5456196be84fd9b0e44ce/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.540834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:44.554599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d633a90bcd14e4da76366cd8812c97d, user_id=0d91a59becd5456196be84fd9b0e44ce, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.557406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.571165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1290/2617] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:44 2022] PUT /identity/v3/projects/8d633a90bcd14e4da76366cd8812c97d/users/0d91a59becd5456196be84fd9b0e44ce/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:44.576180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66498c72-f863-43e3-a79a-1095e506d45f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.576476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66498c72-f863-43e3-a79a-1095e506d45f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.576762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66498c72-f863-43e3-a79a-1095e506d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.595010 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-66498c72-f863-43e3-a79a-1095e506d45f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:44.596207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-66498c72-f863-43e3-a79a-1095e506d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.635521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:44.660658 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1328/2618] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:44.667062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:44.668753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:44.724359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee5b41-12b8-4e02-8f88-384b9fa8eaf2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:44.728076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:44.728720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:44.729043 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:44.729324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:44.739141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:44.740565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:44.750408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1291/2619] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:45.434848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.435847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.436340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.467821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] MFA Rules not processed for user `0d91a59becd5456196be84fd9b0e44ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:45.469139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:45.539433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1329/2620] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:45.544003 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.555414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ba845-96ae-4d1c-880c-056bdfbd8d87 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.560371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.561092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.561401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.562792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.563564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.565522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.599542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1292/2621] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:45.600503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.612389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c148aa-7ebb-41ee-8523-9578a9bd13f8 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.671768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.672451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.672745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.673069 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.673781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.675704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.703865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1330/2622] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:45 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:45.709962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.720429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df1b07-727e-49bd-b78c-d4f9e77387fd admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.726072 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.726923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.727220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.727507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.728358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.729995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.741393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1293/2623] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:45.745661 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.757448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6036e6-6be2-4d19-9bfc-21f28d366ea2 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.762824 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.763696 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.763969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.764256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] PATH_INFO: `/v3/projects/39a39be8a41148fdb6350bc30bd3cc16/users/ab0aa0570c0b40c8931202d2fcb83fed/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.780580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a39be8a41148fdb6350bc30bd3cc16, user_id=ab0aa0570c0b40c8931202d2fcb83fed, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.783316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.797760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1331/2624] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:45 2022] PUT /identity/v3/projects/39a39be8a41148fdb6350bc30bd3cc16/users/ab0aa0570c0b40c8931202d2fcb83fed/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:45.801622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.814235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad04242-3e85-4f77-a968-64409433f29a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.880922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.881624 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.881916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.882199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.882934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.884536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387861c0-0533-4a7a-b188-bc4d2af79b38 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.895083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1294/2625] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:45.900174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:45.911376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59beed72-904c-456e-a9e5-0627e6e63849 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:45.916154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:45.917066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:45.917355 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:45.917635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] PATH_INFO: `/v3/projects/39a39be8a41148fdb6350bc30bd3cc16/users/ab0aa0570c0b40c8931202d2fcb83fed/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:45.919608 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a39be8a41148fdb6350bc30bd3cc16, user_id=ab0aa0570c0b40c8931202d2fcb83fed, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:45.922786 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:45.932896 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1332/2626] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:45 2022] PUT /identity/v3/projects/39a39be8a41148fdb6350bc30bd3cc16/users/ab0aa0570c0b40c8931202d2fcb83fed/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:46.574143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.575065 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.575619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.630932 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] MFA Rules not processed for user `ab0aa0570c0b40c8931202d2fcb83fed`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:46.632239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.682715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:46.720126 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1295/2627] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:46.737263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.739742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.778364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:46.813821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a7d83-ffd3-4aae-9a0a-ae90c8ac118e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.817609 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.818257 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.818678 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.818966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.845040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:46.846754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:46.885146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.886715 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe265e89-77dd-4583-9b71-c97670b704ed service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.897714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1333/2628] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:46.919800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dcfe1-3860-4574-bc86-64a19e7e895b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:46.924656 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:46.925632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:46.925860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:46.926142 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] PATH_INFO: `/v3/users/2b638f188cb942e28930e8a145dc3e2c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:46.927360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b638f188cb942e28930e8a145dc3e2c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:46.929307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:46.979068 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b638f188cb942e28930e8a145dc3e2c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:46.986994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b638f188cb942e28930e8a145dc3e2c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.000742 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b638f188cb942e28930e8a145dc3e2c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.001429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b638f188cb942e28930e8a145dc3e2c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.002037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b638f188cb942e28930e8a145dc3e2c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.014118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1296/2629] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:46 2022] DELETE /identity/v3/users/2b638f188cb942e28930e8a145dc3e2c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:34:47.020027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.020327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.020612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.038093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:47.039351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.106204 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1334/2630] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:47.116916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.118339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.198819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a732236-c1aa-47c4-9cca-bfc869ec648a admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.200336 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.201038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.201330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.201672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.211239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.212683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.216795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:47.225436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1297/2631] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:47.498674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.501203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:47.513903 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d33f1-4197-49ad-ba24-8e5e6296326f None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.519152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.520109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.520465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.520827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] PATH_INFO: `/v3/projects/70cbc11c4bca4ed3a4edcd6cebe8fadf` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.522075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=70cbc11c4bca4ed3a4edcd6cebe8fadf)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.523965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.570893 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70cbc11c4bca4ed3a4edcd6cebe8fadf', 'request_id': 'req-d257199b-9df4-4492-9439-f00d3eff0cdb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.576224 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70cbc11c4bca4ed3a4edcd6cebe8fadf', 'request_id': 'req-d257199b-9df4-4492-9439-f00d3eff0cdb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.582875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70cbc11c4bca4ed3a4edcd6cebe8fadf', 'request_id': 'req-d257199b-9df4-4492-9439-f00d3eff0cdb'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.594223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] The token cache is being invalidate because project 70cbc11c4bca4ed3a4edcd6cebe8fadf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:47.594511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.595146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.596659 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1335/2632] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:47 2022] DELETE /identity/v3/projects/70cbc11c4bca4ed3a4edcd6cebe8fadf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:47.600642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:47.611113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555ad01-5760-4fe0-85e6-a2b9e9690ad3 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:47.669476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:47.670385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:47.670668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:47.671491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] PATH_INFO: `/v3/users/2a9cfcdd0ddb49f8af6bd017d825adeb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:47.672737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9cfcdd0ddb49f8af6bd017d825adeb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:47.674675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:47.731241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9cfcdd0ddb49f8af6bd017d825adeb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.740899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9cfcdd0ddb49f8af6bd017d825adeb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.753649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9cfcdd0ddb49f8af6bd017d825adeb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.754347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9cfcdd0ddb49f8af6bd017d825adeb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.754970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-86fa9558-f7c7-4f6f-abdb-3af4b7d0e88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9cfcdd0ddb49f8af6bd017d825adeb'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:47.768028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1298/2633] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:47 2022] DELETE /identity/v3/users/2a9cfcdd0ddb49f8af6bd017d825adeb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:48.032102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.033583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.042595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:48.047415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257199b-9df4-4492-9439-f00d3eff0cdb admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.112577 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.113539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.113828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.114110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] PATH_INFO: `/v3/projects/b88a74a675634bf9b936d38c5cda6cf8` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.115324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88a74a675634bf9b936d38c5cda6cf8)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.117275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.163102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88a74a675634bf9b936d38c5cda6cf8', 'request_id': 'req-644af9c2-e219-46ba-8010-37481c9f48c7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.168392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88a74a675634bf9b936d38c5cda6cf8', 'request_id': 'req-644af9c2-e219-46ba-8010-37481c9f48c7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.176228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88a74a675634bf9b936d38c5cda6cf8', 'request_id': 'req-644af9c2-e219-46ba-8010-37481c9f48c7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.187916 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] The token cache is being invalidate because project b88a74a675634bf9b936d38c5cda6cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:34:48.188206 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.188835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:48.190392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1336/2634] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:34:48 2022] DELETE /identity/v3/projects/b88a74a675634bf9b936d38c5cda6cf8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:48.195423 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce030-61bd-4280-bae3-1436f07c1068 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.195714 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce030-61bd-4280-bae3-1436f07c1068 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.196070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ce030-61bd-4280-bae3-1436f07c1068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.214490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-786ce030-61bd-4280-bae3-1436f07c1068 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:48.215725 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-786ce030-61bd-4280-bae3-1436f07c1068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.257739 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.260150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.279529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1299/2635] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:48.288800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.289133 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.289427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.307900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:48.309152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.317511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1300/2636] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:34:48.318486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644af9c2-e219-46ba-8010-37481c9f48c7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.321567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.322095 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.322744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.323032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.323312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.325076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.331083 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8603fd-e99d-4414-96c2-cb5f8fee9fe0 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.335679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.336348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.336623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.336929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.337663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:34:48.337979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.339582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.346871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:48.354144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1301/2637] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:34:48.357817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.368594 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa815a49-63a0-48e3-8e1f-eb566ae8ee91 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.373402 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.374046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.374328 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.374716 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.375410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.376144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:48.377312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.395028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.396649 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.404968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1302/2638] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:48.409390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.410358 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1337/2639] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:48.477919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408284-6a25-479c-81f6-b28775d8ffa6 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.482812 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.484117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.484679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.484999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.485698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.487717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.510911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1303/2640] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:34:48.512147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.523987 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe482a-d5e5-407b-9e5f-40e996b9155d service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.528632 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.529539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.529850 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.532351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.533148 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.534757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.545167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1338/2641] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:34:48.549730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.559848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada3af8-b967-425a-bd10-8934ebbd73b4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.565730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.566417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.566715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.567002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.567750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.569392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad9ddc-ff6d-4be0-ae05-910334a7b475 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.585619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1304/2642] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:48.586513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.597412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cb92-8279-4056-826c-4df4593a207f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.602141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.603141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.603436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.603714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] PATH_INFO: `/v3/projects/d088f76ae95943dea3003de3adbdd339/users/bdbda16da0c3484794a88cdcb056e2d0/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.621136 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d088f76ae95943dea3003de3adbdd339, user_id=bdbda16da0c3484794a88cdcb056e2d0, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.623815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.637221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1339/2643] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:48 2022] PUT /identity/v3/projects/d088f76ae95943dea3003de3adbdd339/users/bdbda16da0c3484794a88cdcb056e2d0/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:48.647578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.648069 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.648187 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.676999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] MFA Rules not processed for user `bdbda16da0c3484794a88cdcb056e2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:48.678164 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:48.745155 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1305/2644] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:48.749809 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.815835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c751-6560-4d64-98bd-240ecf6b7726 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.820835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.821538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.821820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.822105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.822810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.824700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.851745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1340/2645] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:48.855907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.867946 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9118401-65a5-4de7-bf16-ff90ff4f2466 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.910675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:48.927090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.927686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.927974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.928256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.928979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.930926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.952444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1306/2646] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:34:48 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:34:48.956664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:48.966973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31440c43-95c3-44db-94f4-62c1c1cd4b7b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:48.975806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:48.976506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:48.976794 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:48.977104 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:48.977838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:48.979434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:48.990900 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1341/2647] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:48.995063 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.004912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f5773-0658-49f3-af6e-00f81045b67d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.009405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.010276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.010558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.010831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] PATH_INFO: `/v3/projects/3725cbd211594746935930b46899c27e/users/80c967d08c5a434aaf310d56a52dd76e/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.026017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3725cbd211594746935930b46899c27e, user_id=80c967d08c5a434aaf310d56a52dd76e, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.028662 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.041986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1307/2648] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:48 2022] PUT /identity/v3/projects/3725cbd211594746935930b46899c27e/users/80c967d08c5a434aaf310d56a52dd76e/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:49.045958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.056310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79ffc8-790a-4c35-9d2b-bd9074809de4 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.114408 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.115084 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.115375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.115657 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.116385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.118036 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23446ec-4f0b-4908-8263-66b9af9ed4aa admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.128465 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1342/2649] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:34:49.132541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.144042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c127f-5bf8-4f5e-a300-0ccc56dafad5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.148649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.149562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.149856 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.150135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] PATH_INFO: `/v3/projects/3725cbd211594746935930b46899c27e/users/80c967d08c5a434aaf310d56a52dd76e/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.152093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3725cbd211594746935930b46899c27e, user_id=80c967d08c5a434aaf310d56a52dd76e, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.154769 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.165392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1308/2650] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:34:49 2022] PUT /identity/v3/projects/3725cbd211594746935930b46899c27e/users/80c967d08c5a434aaf310d56a52dd76e/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:49.176810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.177191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.177426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.204978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] MFA Rules not processed for user `80c967d08c5a434aaf310d56a52dd76e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:49.206187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:49.214122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:49.275195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1343/2651] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:34:49.286931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.343075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7379341-70c1-4b14-ba3c-aa2802fda34e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.346797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.347472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.347765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.348046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.357203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.359327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.368713 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1309/2652] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:34:49.440073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:49.449504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7db7d-f9d8-4422-8294-614f303d7121 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:49.454097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:49.454762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:49.455048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:49.455327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:49.514168 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:49.516384 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3110cf-94e9-48ab-89da-f88fc93b53ce service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:49.525420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1344/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:34:59.364481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=50,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:59.521340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=53 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:34:59.741209 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:34:59.808569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd5bdb-a7a3-4f00-a77d-5e7776321492 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:34:59.813178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:34:59.814051 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.814335 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.814617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] PATH_INFO: `/v3/users/bdbda16da0c3484794a88cdcb056e2d0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.815830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbda16da0c3484794a88cdcb056e2d0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:34:59.817797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:34:59.870988 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbda16da0c3484794a88cdcb056e2d0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.878749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbda16da0c3484794a88cdcb056e2d0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.892764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbda16da0c3484794a88cdcb056e2d0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.893510 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbda16da0c3484794a88cdcb056e2d0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.894141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbda16da0c3484794a88cdcb056e2d0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:34:59.907278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1310/2654] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:34:59 2022] DELETE /identity/v3/users/bdbda16da0c3484794a88cdcb056e2d0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:34:59.911880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:34:59.912177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:34:59.912463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:34:59.929565 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:34:59.930841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:34:59.990842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1345/2655] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:00.000363 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:00.001801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:00.059310 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715d70e-4b42-44e7-adb1-f3b06f3e509f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:00.063168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:00.063820 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:00.064108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:00.064389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:00.073317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:00.074738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:00.084928 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1311/2656] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:00.361229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:00.362688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:00.376222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be060b52-d78e-4a57-9821-f81677b96737 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:00.381177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:00.382044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:00.382333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:00.382614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] PATH_INFO: `/v3/projects/d088f76ae95943dea3003de3adbdd339` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:00.383863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d088f76ae95943dea3003de3adbdd339)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:00.385807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:00.432971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd088f76ae95943dea3003de3adbdd339', 'request_id': 'req-2bb700f5-163b-4b9a-a640-1dc7727d54d5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.439115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd088f76ae95943dea3003de3adbdd339', 'request_id': 'req-2bb700f5-163b-4b9a-a640-1dc7727d54d5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.446131 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd088f76ae95943dea3003de3adbdd339', 'request_id': 'req-2bb700f5-163b-4b9a-a640-1dc7727d54d5'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.458244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] The token cache is being invalidate because project d088f76ae95943dea3003de3adbdd339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:00.458533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.459151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.460669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1346/2657] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:00 2022] DELETE /identity/v3/projects/d088f76ae95943dea3003de3adbdd339 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:00.464611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:00.474849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d02f0-2019-4e2d-9946-633e15814902 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:00.495050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:00.537246 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:00.538129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:00.538419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:00.538701 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] PATH_INFO: `/v3/users/80c967d08c5a434aaf310d56a52dd76e` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:00.539901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c967d08c5a434aaf310d56a52dd76e)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:00.541974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:00.592088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c967d08c5a434aaf310d56a52dd76e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.601048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c967d08c5a434aaf310d56a52dd76e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.615590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c967d08c5a434aaf310d56a52dd76e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.616229 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c967d08c5a434aaf310d56a52dd76e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.617028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-dd15f8af-8e1a-454f-8fff-5e9bafb79032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c967d08c5a434aaf310d56a52dd76e'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:00.634161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1312/2658] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:00 2022] DELETE /identity/v3/users/80c967d08c5a434aaf310d56a52dd76e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:00.934061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:00.935573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:00.949983 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb700f5-163b-4b9a-a640-1dc7727d54d5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.014162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.015811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.016105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.016392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] PATH_INFO: `/v3/projects/3725cbd211594746935930b46899c27e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.017619 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3725cbd211594746935930b46899c27e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.019523 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.052409 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.070151 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3725cbd211594746935930b46899c27e', 'request_id': 'req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.075773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3725cbd211594746935930b46899c27e', 'request_id': 'req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.083585 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3725cbd211594746935930b46899c27e', 'request_id': 'req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.095477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] The token cache is being invalidate because project 3725cbd211594746935930b46899c27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:01.095762 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.096386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5195eb3d-2d7e-424c-9d3d-2a9c029d351e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.098007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1347/2659] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:00 2022] DELETE /identity/v3/projects/3725cbd211594746935930b46899c27e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:01.104333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.104633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.104976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.123449 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:01.124672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.186110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1313/2660] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:01.194149 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.194444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.194725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.212727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:01.213959 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.222662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1348/2661] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:01.226855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.236935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622be54-55c1-481a-a1e4-6c6b0227481a None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.241413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.242066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.242352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.242632 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.243354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:01.243665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.245278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.256698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1314/2662] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:01.260779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.270680 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42881a6a-3f0f-4476-9ed5-0894017b0438 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.275266 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.275913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.276202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.276481 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.277230 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.279124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.305142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1349/2663] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:01.309356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.319096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c96a0b-b85e-4d22-a3a4-578a74c0ffd5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.344621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.374919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.375577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.375861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.376136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.376831 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.378774 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.398452 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1315/2664] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:01.403103 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.414181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a512aa-ec6a-49d1-9e08-056df6a933c1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.418703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.419356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.419643 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.419921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.420730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.422442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.432676 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1350/2665] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:01.437040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.447631 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1bcee-18a2-47a1-a155-81821857dbd2 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.453530 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.454208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.454499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.454784 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.456343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.457173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e39130-6f69-4baf-be51-0661cb0692a0 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.469647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1316/2666] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:01.473965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.484554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d15cc9-0da6-4636-976b-d1a217c0a57e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.489383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.490138 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.490426 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.490707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] PATH_INFO: `/v3/projects/9b5dff173a734a3cb4c7bfd552a2ba88/users/e17efb409b5740ab9da500d7129b64c6/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.505986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5dff173a734a3cb4c7bfd552a2ba88, user_id=e17efb409b5740ab9da500d7129b64c6, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.508617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.522145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1351/2667] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:01 2022] PUT /identity/v3/projects/9b5dff173a734a3cb4c7bfd552a2ba88/users/e17efb409b5740ab9da500d7129b64c6/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:01.533517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.533809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.534095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.560301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] MFA Rules not processed for user `e17efb409b5740ab9da500d7129b64c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:01.561633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.626790 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1317/2668] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:01.638316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.640871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.698418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee64256-f5f0-49fc-9bb8-3fc55ec2bbf5 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.702525 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.703197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.703486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.703760 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.712545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.714004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c07fe-61a9-4035-8da5-411cc50c95f3 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.723536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1352/2669] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:01.743251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:01.807923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce5231-30de-4dd6-a205-0956412f8b2d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:01.813667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:01.814577 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.814861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.815140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] PATH_INFO: `/v3/users/e17efb409b5740ab9da500d7129b64c6` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.816348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17efb409b5740ab9da500d7129b64c6)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:01.818299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:01.862153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.869538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17efb409b5740ab9da500d7129b64c6'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.878538 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17efb409b5740ab9da500d7129b64c6'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.895032 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17efb409b5740ab9da500d7129b64c6'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.895754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17efb409b5740ab9da500d7129b64c6'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.896382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17efb409b5740ab9da500d7129b64c6'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:01.909491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1318/2670] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:01 2022] DELETE /identity/v3/users/e17efb409b5740ab9da500d7129b64c6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:01.913956 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:01.914251 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:01.914536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:01.932049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:01.933271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:01.982020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:01.999430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1353/2671] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:02.006304 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.009295 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.070793 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e014f-658f-4c26-b33a-c9b52e78de78 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.074710 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.075365 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.075653 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.075936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.086889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.088617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1773d3-747f-4de1-ad23-7169fac82609 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.098658 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1319/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:02.384995 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.386495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.398635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca10c-69c8-40ec-9a69-ed4e4b576c4b None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.403276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.404147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.404438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.404722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] PATH_INFO: `/v3/projects/9b5dff173a734a3cb4c7bfd552a2ba88` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.406048 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5dff173a734a3cb4c7bfd552a2ba88)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.407971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.455137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5dff173a734a3cb4c7bfd552a2ba88', 'request_id': 'req-66588537-722e-4f3b-a7ba-d7da58d16f03'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.460646 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5dff173a734a3cb4c7bfd552a2ba88', 'request_id': 'req-66588537-722e-4f3b-a7ba-d7da58d16f03'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.467939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5dff173a734a3cb4c7bfd552a2ba88', 'request_id': 'req-66588537-722e-4f3b-a7ba-d7da58d16f03'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.479178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] The token cache is being invalidate because project 9b5dff173a734a3cb4c7bfd552a2ba88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:02.479406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.480044 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-66588537-722e-4f3b-a7ba-d7da58d16f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:02.481627 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1354/2673] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:02 2022] DELETE /identity/v3/projects/9b5dff173a734a3cb4c7bfd552a2ba88 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:02.487165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.487655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.487818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.507143 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:02.508370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.573475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1320/2674] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:02.582741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.583046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.583430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.602508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:02.603728 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:02.612412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1355/2675] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:02.616618 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.626762 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523200d-fe55-4605-82ef-2b1343fc7605 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.631479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.632140 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.632427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.632706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.633480 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:02.633792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.635531 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.649484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1321/2676] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:02.653840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.664400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87995aec-6642-4b5c-9abf-a384d90b9e99 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.669195 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.669861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.670154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.670436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.671793 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.673875 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.698805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1356/2677] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:02 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:02.703117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.713097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30de0-3d6d-490b-acb3-3057df358450 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.757275 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:02.771898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.772599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.772901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.773188 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.773894 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.775828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.798771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1322/2678] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:02 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:02.800002 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.810279 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f599a-d37d-4831-8a22-07b419841aae admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.814939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.815605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.815897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.816182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.816949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.818554 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.830115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1357/2679] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:02.834115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.844696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81e4a-b448-430d-8833-31d875f21006 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.849309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.849972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.850265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.850551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.851293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.852927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168165d3-de5a-4148-a46d-c76cd2fb9381 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.863781 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1323/2680] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:02.867299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:02.877365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3b5f-743c-4b1c-a39b-8961f8049968 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:02.882237 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:02.883172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.883478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.883745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] PATH_INFO: `/v3/projects/3eef420a490c47da82d74a67eca42531/users/da1f463e09b74ef0a53a03a4e6779fdf/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.904090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eef420a490c47da82d74a67eca42531, user_id=da1f463e09b74ef0a53a03a4e6779fdf, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:02.907650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:02.920476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1358/2681] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:02 2022] PUT /identity/v3/projects/3eef420a490c47da82d74a67eca42531/users/da1f463e09b74ef0a53a03a4e6779fdf/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:02.930905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:02.931202 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:02.931511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:02.957646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] MFA Rules not processed for user `da1f463e09b74ef0a53a03a4e6779fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:02.958847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:03.025250 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1324/2682] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:03.029775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.093424 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab63eb3-ac16-4788-9792-23f6d601915e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.098091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.098892 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.099188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.099471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.100161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.102178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.121393 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:03.130413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1359/2683] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:03.134643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.146490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e2872-fa6b-442d-ba34-4042fcb4be09 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.203511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.204198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.204476 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.204761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.205492 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.207410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.227197 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1325/2684] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:03.231315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.242130 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a682a-fc98-46e3-8413-e56ad72cb59f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.246682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.247347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.247635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.247918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.248694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.250844 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.261221 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1360/2685] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:03.266096 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.276048 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24a7b8-f9e4-48c8-8cb9-3f3fccfb7460 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.281358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.282036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.282303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.282728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.283330 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.284916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfa2db-41a3-42ab-8921-602909d2964f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.294830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1326/2686] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:03.298978 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.308923 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872ff95-5449-4472-a362-7a64b5e24997 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.314174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.315059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.315348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.315635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] PATH_INFO: `/v3/projects/b12c328cced44efb842cbdfe4067e291/users/2961974df8064ac381f23e15306c5fe9/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.330670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12c328cced44efb842cbdfe4067e291, user_id=2961974df8064ac381f23e15306c5fe9, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.333357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.345936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1361/2687] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:03 2022] PUT /identity/v3/projects/b12c328cced44efb842cbdfe4067e291/users/2961974df8064ac381f23e15306c5fe9/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:03.355802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.356131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.356412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.383050 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] MFA Rules not processed for user `2961974df8064ac381f23e15306c5fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:03.384248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:03.405930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:03.456871 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1327/2688] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:03.461273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.528758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06fb2e-7935-4b9b-8b39-77015773c3bd admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.540015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.540801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.541232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.541547 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.542674 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.545191 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.576569 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1362/2689] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:03.578450 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.589963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bca198-7a03-4ad5-b917-4569fafccf6f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.650435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.651120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.651411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.651696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.652401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.654417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.678973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1328/2690] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:03.680273 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.691308 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35765e2-e07a-4e79-bddb-b3b6f7cabe84 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.696018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.696683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.697006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.697291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.698051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.699641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.712364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1363/2691] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:03.715462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.727400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba10a9e-030a-4560-a4ea-39fb549d9e4f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.732755 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.733816 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.734171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.734458 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] PATH_INFO: `/v3/projects/568f001ca8f74667acafd8f9e44fe68d/users/4a0df885bd3f478583baf64bafb3e838/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.751557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568f001ca8f74667acafd8f9e44fe68d, user_id=4a0df885bd3f478583baf64bafb3e838, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.754659 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.773834 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1329/2692] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:03 2022] PUT /identity/v3/projects/568f001ca8f74667acafd8f9e44fe68d/users/4a0df885bd3f478583baf64bafb3e838/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:03.777703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.789296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7fee0-df1b-4be7-b03b-fcef08160ae0 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.858229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.859006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.859293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.859579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.860320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.861969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd151c-a6db-4715-ae67-1acb9742c5a1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.872394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1364/2693] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:03.876580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:03.886736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07943e-da97-4649-a478-2bc771d38880 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:03.893285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:03.894324 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.894605 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.894886 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] PATH_INFO: `/v3/projects/568f001ca8f74667acafd8f9e44fe68d/users/4a0df885bd3f478583baf64bafb3e838/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.897482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568f001ca8f74667acafd8f9e44fe68d, user_id=4a0df885bd3f478583baf64bafb3e838, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:03.899668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:03.911851 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1330/2694] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:03 2022] PUT /identity/v3/projects/568f001ca8f74667acafd8f9e44fe68d/users/4a0df885bd3f478583baf64bafb3e838/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:03.923458 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:03.923757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:03.924108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:03.952466 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] MFA Rules not processed for user `4a0df885bd3f478583baf64bafb3e838`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:03.953719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:03.987740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:04.059478 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1365/2695] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:04.084213 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:04.087214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:04.161077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ae18-a84e-4b07-9288-6be13fe10280 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:04.165655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:04.166598 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:04.167037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:04.167468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:04.206668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:04.247681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:04.249273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:04.261822 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1331/2696] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:04.626966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:04.629028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:04.698503 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc38883-1c90-42b5-acba-c35d0d57cd29 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:04.703146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:04.704049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:04.704339 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:04.704623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] PATH_INFO: `/v3/users/0d91a59becd5456196be84fd9b0e44ce` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:04.705879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d91a59becd5456196be84fd9b0e44ce)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:04.707873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:04.754177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d91a59becd5456196be84fd9b0e44ce'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:04.761373 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d91a59becd5456196be84fd9b0e44ce'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:04.768918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d91a59becd5456196be84fd9b0e44ce'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:04.769546 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d91a59becd5456196be84fd9b0e44ce'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:04.782187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d91a59becd5456196be84fd9b0e44ce'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:04.788108 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1366/2697] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:04 2022] DELETE /identity/v3/users/0d91a59becd5456196be84fd9b0e44ce => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:04.964975 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:04.977179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69d19-bcde-4315-a429-64251a4206d0 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.036473 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.037356 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.037642 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.037920 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] PATH_INFO: `/v3/projects/8d633a90bcd14e4da76366cd8812c97d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.039080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d633a90bcd14e4da76366cd8812c97d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.040990 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.085889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d633a90bcd14e4da76366cd8812c97d', 'request_id': 'req-c2a564b8-2850-40ba-aa62-df52c2887627'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.091963 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d633a90bcd14e4da76366cd8812c97d', 'request_id': 'req-c2a564b8-2850-40ba-aa62-df52c2887627'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.098576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d633a90bcd14e4da76366cd8812c97d', 'request_id': 'req-c2a564b8-2850-40ba-aa62-df52c2887627'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.111401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] The token cache is being invalidate because project 8d633a90bcd14e4da76366cd8812c97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:05.111694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.112321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.113889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1332/2698] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:04 2022] DELETE /identity/v3/projects/8d633a90bcd14e4da76366cd8812c97d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:05.117960 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.119387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.132744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd071a-990b-42c8-819c-3ca789961f39 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.168200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:05.192021 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.192912 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.193197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.193471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] PATH_INFO: `/v3/users/ab0aa0570c0b40c8931202d2fcb83fed` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.194640 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0aa0570c0b40c8931202d2fcb83fed)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.196531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.239775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0aa0570c0b40c8931202d2fcb83fed'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.246553 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0aa0570c0b40c8931202d2fcb83fed'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.253859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0aa0570c0b40c8931202d2fcb83fed'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.254497 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0aa0570c0b40c8931202d2fcb83fed'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.267210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0aa0570c0b40c8931202d2fcb83fed'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.272997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1367/2699] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:05 2022] DELETE /identity/v3/users/ab0aa0570c0b40c8931202d2fcb83fed => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:05.692090 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.695441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.730934 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a564b8-2850-40ba-aa62-df52c2887627 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.796916 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.797792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.798104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.798389 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] PATH_INFO: `/v3/projects/39a39be8a41148fdb6350bc30bd3cc16` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.799929 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a39be8a41148fdb6350bc30bd3cc16)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.800051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:05.801412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.801964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.803546 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:05.866779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a39be8a41148fdb6350bc30bd3cc16', 'request_id': 'req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.868911 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea1a3e-2165-48f5-bf90-80c643443df1 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:05.873219 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a39be8a41148fdb6350bc30bd3cc16', 'request_id': 'req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.880004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a39be8a41148fdb6350bc30bd3cc16', 'request_id': 'req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.893685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] The token cache is being invalidate because project 39a39be8a41148fdb6350bc30bd3cc16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:05.893979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.894606 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cdfa277f-e0eb-4ad5-96d4-d87c705bd3ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:05.896144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1333/2700] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:05 2022] DELETE /identity/v3/projects/39a39be8a41148fdb6350bc30bd3cc16 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:05.909868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3b2d-a921-43ab-b115-8ae72ce2d73d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.910172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3b2d-a921-43ab-b115-8ae72ce2d73d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.910459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3b2d-a921-43ab-b115-8ae72ce2d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.923536 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:05.924190 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:05.924477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:05.924757 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:05.926605 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.929483 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-ceec3b2d-a921-43ab-b115-8ae72ce2d73d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:05.930666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-ceec3b2d-a921-43ab-b115-8ae72ce2d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:05.953451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:05.988189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:05.989641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:05.993345 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1334/2701] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:05.999395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1368/2702] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:06.002233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.002525 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.002977 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.022333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:06.023563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.032385 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1335/2703] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:06.036670 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.047950 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dde36-6cee-4f41-97cb-14dd1774e528 service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.052451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.053160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.053450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.053730 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.054463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:06.054774 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.056370 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.068833 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1369/2704] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:06.073556 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.083596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08bc9-149f-4d77-8be1-145f0af1f54e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.088381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.089182 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.089477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.089765 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.090466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.093020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.125007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1336/2705] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:06.126186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.136202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa437-436f-481e-ac6b-e1f1c17b3e65 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.204014 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.204925 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.205289 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.205702 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.206560 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.208699 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.230003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1370/2706] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:06.234753 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.245247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc5d9-149d-4889-8dfb-752d6fff89b8 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.251081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.251748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.252039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.252321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.253092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.254700 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.266326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1337/2707] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:06.270539 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.281160 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bce35-a1a3-400a-afce-8e05a9f1391d admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.287042 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.287719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.288012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.288291 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.289065 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.290703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dded0-d1ce-433d-8810-82f3361e32a7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.302392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1371/2708] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:06.306813 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.318306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b04b7-6fbe-4f69-90d9-c88c0970cc64 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.324017 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.325025 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.325306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.325611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] PATH_INFO: `/v3/projects/3497b122beac409c9b84da546d3ce823/users/e222e58b9ded4129bff2a55040b1dbb7/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.341117 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3497b122beac409c9b84da546d3ce823, user_id=e222e58b9ded4129bff2a55040b1dbb7, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.345037 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.359269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1338/2709] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:06 2022] PUT /identity/v3/projects/3497b122beac409c9b84da546d3ce823/users/e222e58b9ded4129bff2a55040b1dbb7/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:06.370500 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.370798 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.371085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.399620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] MFA Rules not processed for user `e222e58b9ded4129bff2a55040b1dbb7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:06.400846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.467819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1372/2710] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:06.497858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.500154 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.526542 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.528012 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.532673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:06.556958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:06.561945 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047a986-62ed-450b-baa2-9f412d01c1d4 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.565751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.566445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.566730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.567011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.575570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.577256 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.586283 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1339/2711] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:06.590451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41475b8-f402-43c2-8b87-9bee0be307a3 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.595207 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.595866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.596178 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.596459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.598263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.601076 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.667775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.668709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c986c5d-5257-4251-9b70-4981462afb8c service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.669322 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b2ee8-32f5-417c-a7cd-53a7627b4a5c service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.673388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.674357 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.674646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.674996 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] PATH_INFO: `/v3/users/e222e58b9ded4129bff2a55040b1dbb7` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.676327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e222e58b9ded4129bff2a55040b1dbb7)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.678493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.679118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1373/2712] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:06.730208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e222e58b9ded4129bff2a55040b1dbb7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:06.741721 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e222e58b9ded4129bff2a55040b1dbb7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:06.756902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e222e58b9ded4129bff2a55040b1dbb7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:06.757911 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e222e58b9ded4129bff2a55040b1dbb7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:06.758582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e222e58b9ded4129bff2a55040b1dbb7'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:06.772706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1340/2713] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:06 2022] DELETE /identity/v3/users/e222e58b9ded4129bff2a55040b1dbb7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:06.777205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.777498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.777779 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.802488 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:06.803684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.863756 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1374/2714] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:06.873503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:06.875040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:06.946689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:06.950914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feee901-cd07-4879-a3a0-0e5adf4214a8 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:06.957360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:06.958226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:06.958540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:06.958987 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:06.970135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:06.971599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495097cd-4b95-4d0a-a208-c69918bb1ebd service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:06.982684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1341/2715] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:07.250604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.252162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.265177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922e991-3433-4b52-8567-96fb1220bf43 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.271170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.272046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.272333 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.272961 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] PATH_INFO: `/v3/projects/3497b122beac409c9b84da546d3ce823` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.274117 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3497b122beac409c9b84da546d3ce823)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.276334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.324604 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3497b122beac409c9b84da546d3ce823', 'request_id': 'req-637439ed-188b-454a-88e2-e5bed3bca296'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:07.330761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3497b122beac409c9b84da546d3ce823', 'request_id': 'req-637439ed-188b-454a-88e2-e5bed3bca296'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:07.338294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3497b122beac409c9b84da546d3ce823', 'request_id': 'req-637439ed-188b-454a-88e2-e5bed3bca296'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:07.340211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:07.351159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] The token cache is being invalidate because project 3497b122beac409c9b84da546d3ce823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:07.351456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:07.352094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-637439ed-188b-454a-88e2-e5bed3bca296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:07.353664 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1375/2716] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:07 2022] DELETE /identity/v3/projects/3497b122beac409c9b84da546d3ce823 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:07.358687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.359179 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.359273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.376795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:07.378035 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.437835 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1342/2717] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:07.499858 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.500073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.500314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.520805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:07.522062 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.529941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1376/2718] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:07.534115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.543655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9d89c-42a6-409d-ab6b-122b3ac1edc5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.548815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.549497 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.549782 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.550057 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.550775 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:07.551091 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.552668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.564541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1343/2719] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:07.568688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.577857 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6809d-3dbf-48d1-a1ca-b1ae1c213c9a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.582439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.583082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.583366 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.583644 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.584342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.586285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.611209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1377/2720] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:07.615255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.625682 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db30f4-0e13-4b6b-80b4-9b364a9b3d16 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.682186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.682857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.683152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.683418 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.684141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.686875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.705562 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1344/2721] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:07.709884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.720666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dcfb-54e6-400b-a491-836b1a5a0d5f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.725512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.726187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.726474 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.726748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.727516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.729164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.740476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1378/2722] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:07.744470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.754982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412ded6-9ad7-48ff-8e80-93352bcdc514 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.759424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.760077 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.760364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.760644 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.761412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.762999 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff02f5-5d67-46a4-9312-6f1c7ecbfe51 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.773249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1345/2723] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:07.777211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.788771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584870a-f815-4278-8dba-e959d8b45075 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:07.796357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:07.797399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.798077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.798077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] PATH_INFO: `/v3/projects/a89c43430bef4e3a89eea843d2f3d390/users/9b5258a675f94710929b8ab469afcfa8/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.815249 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89c43430bef4e3a89eea843d2f3d390, user_id=9b5258a675f94710929b8ab469afcfa8, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:07.818395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:07.832669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1379/2724] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:07 2022] PUT /identity/v3/projects/a89c43430bef4e3a89eea843d2f3d390/users/9b5258a675f94710929b8ab469afcfa8/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:07.843394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:07.843558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:07.843736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:07.870400 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] MFA Rules not processed for user `9b5258a675f94710929b8ab469afcfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:07.871610 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:07.927709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:07.949621 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1346/2725] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:07.960738 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:07.962688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:08.030276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8155e40-c69d-4aeb-9d71-ecd45d5f69fe admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:08.034245 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:08.034897 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:08.035180 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:08.035459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:08.044200 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:08.046444 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:08.055906 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1380/2726] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:10.551593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:10.553030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:10.601377 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:10.620495 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876361e-fccf-43dc-8bf4-d49441c58c7b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:10.625974 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:10.626853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.627144 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.627429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] PATH_INFO: `/v3/users/da1f463e09b74ef0a53a03a4e6779fdf` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.628627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1f463e09b74ef0a53a03a4e6779fdf)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:10.630590 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:10.666989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd672-1e3f-4873-b6cd-aaac2c8f778f service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:10.674202 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:10.675147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.675472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.675763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] PATH_INFO: `/v3/users/9b5258a675f94710929b8ab469afcfa8` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.677177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5258a675f94710929b8ab469afcfa8)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:10.679174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:10.683649 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1f463e09b74ef0a53a03a4e6779fdf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.692099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1f463e09b74ef0a53a03a4e6779fdf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.717390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1f463e09b74ef0a53a03a4e6779fdf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.718190 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1f463e09b74ef0a53a03a4e6779fdf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.718994 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-66436a71-a601-4dc2-86aa-e7a0d6208571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1f463e09b74ef0a53a03a4e6779fdf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.731740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1347/2727] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:10 2022] DELETE /identity/v3/users/da1f463e09b74ef0a53a03a4e6779fdf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:10.733513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:10.736926 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.737159 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.737444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.746183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5258a675f94710929b8ab469afcfa8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.756665 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:10.757005 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5258a675f94710929b8ab469afcfa8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.757936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:10.766606 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5258a675f94710929b8ab469afcfa8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.767314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5258a675f94710929b8ab469afcfa8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.780471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-2ec40096-e282-4ab0-b715-55693a22c402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5258a675f94710929b8ab469afcfa8'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:10.786535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1381/2728] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:10 2022] DELETE /identity/v3/users/9b5258a675f94710929b8ab469afcfa8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:10.790860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.791156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.791437 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.809156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:10.810375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:10.822634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1348/2729] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:10.834535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:10.835953 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:10.848039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:10.878305 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1382/2730] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:10.887886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:10.889600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:10.901616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d608dc-2041-4c4b-b0ee-b7cc6c71d758 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:10.947580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e989d12-ec6b-4e1d-93e6-ca7c4489457a None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:10.949576 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:10.950247 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.950536 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.950821 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.951330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:10.952034 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:10.952434 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:10.952716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:10.960405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:10.961876 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:10.961996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:10.963364 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:10.972454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1349/2731] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:10.973750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1383/2732] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:11.343773 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:11.345370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:11.359369 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142e5f1-2975-4f9c-b0a7-37063a5bb187 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:11.365031 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:11.365927 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:11.366217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:11.366668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] PATH_INFO: `/v3/projects/3eef420a490c47da82d74a67eca42531` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:11.367918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eef420a490c47da82d74a67eca42531)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:11.369857 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:11.382436 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:11.384455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:11.405314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe07bc-c718-4b2f-be58-481dd6a8ffc0 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:11.411193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:11.412004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:11.412298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:11.412532 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] PATH_INFO: `/v3/projects/a89c43430bef4e3a89eea843d2f3d390` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:11.413783 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89c43430bef4e3a89eea843d2f3d390)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:11.415792 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:11.433593 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eef420a490c47da82d74a67eca42531', 'request_id': 'req-5bf93275-9a79-4642-be49-f2896e3ff3ec'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.439925 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eef420a490c47da82d74a67eca42531', 'request_id': 'req-5bf93275-9a79-4642-be49-f2896e3ff3ec'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.446170 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eef420a490c47da82d74a67eca42531', 'request_id': 'req-5bf93275-9a79-4642-be49-f2896e3ff3ec'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.466193 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89c43430bef4e3a89eea843d2f3d390', 'request_id': 'req-059dfa19-78f6-4877-9313-552f011f7e76'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.468567 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] The token cache is being invalidate because project 3eef420a490c47da82d74a67eca42531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:11.469736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.469736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.472092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:11.475705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1350/2733] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:11 2022] DELETE /identity/v3/projects/3eef420a490c47da82d74a67eca42531 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:11.476645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89c43430bef4e3a89eea843d2f3d390', 'request_id': 'req-059dfa19-78f6-4877-9313-552f011f7e76'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.481802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:11.483893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:11.486325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89c43430bef4e3a89eea843d2f3d390', 'request_id': 'req-059dfa19-78f6-4877-9313-552f011f7e76'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.500590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] The token cache is being invalidate because project a89c43430bef4e3a89eea843d2f3d390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:11.500590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.500954 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.502740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1384/2734] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:11 2022] DELETE /identity/v3/projects/a89c43430bef4e3a89eea843d2f3d390 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:11.554553 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:11.573588 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf93275-9a79-4642-be49-f2896e3ff3ec admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:11.636358 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:11.637278 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:11.637564 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:11.637854 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] PATH_INFO: `/v3/users/2961974df8064ac381f23e15306c5fe9` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:11.639724 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2961974df8064ac381f23e15306c5fe9)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:11.641937 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:11.694668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2961974df8064ac381f23e15306c5fe9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.705903 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2961974df8064ac381f23e15306c5fe9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.720279 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2961974df8064ac381f23e15306c5fe9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.721080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2961974df8064ac381f23e15306c5fe9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.726399 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2961974df8064ac381f23e15306c5fe9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:11.735217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1351/2735] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:11 2022] DELETE /identity/v3/users/2961974df8064ac381f23e15306c5fe9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:12.078788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:12.080463 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:12.098228 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dfa19-78f6-4877-9313-552f011f7e76 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:12.174306 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:12.175347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.175651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.175939 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] PATH_INFO: `/v3/projects/b12c328cced44efb842cbdfe4067e291` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.177181 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b12c328cced44efb842cbdfe4067e291)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:12.179089 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:12.227096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12c328cced44efb842cbdfe4067e291', 'request_id': 'req-210d89c9-ba09-4aa0-a3d5-688476011c7b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.233935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12c328cced44efb842cbdfe4067e291', 'request_id': 'req-210d89c9-ba09-4aa0-a3d5-688476011c7b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.241018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12c328cced44efb842cbdfe4067e291', 'request_id': 'req-210d89c9-ba09-4aa0-a3d5-688476011c7b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.253365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] The token cache is being invalidate because project b12c328cced44efb842cbdfe4067e291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:12.253684 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.254353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.256508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1385/2736] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:12 2022] DELETE /identity/v3/projects/b12c328cced44efb842cbdfe4067e291 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:12.260468 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:12.262015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:12.275290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c166cff-e804-4907-9925-b768847ce1ce admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:12.334917 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:12.335818 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.336104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.336381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] PATH_INFO: `/v3/users/4a0df885bd3f478583baf64bafb3e838` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.337627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0df885bd3f478583baf64bafb3e838)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:12.339563 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:12.356541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:12.393991 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0df885bd3f478583baf64bafb3e838'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.401681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0df885bd3f478583baf64bafb3e838'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.414982 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0df885bd3f478583baf64bafb3e838'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.415652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0df885bd3f478583baf64bafb3e838'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.416277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cff83521-aa1b-4ac0-9f75-324a4e042fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0df885bd3f478583baf64bafb3e838'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.430440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1352/2737] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:12 2022] DELETE /identity/v3/users/4a0df885bd3f478583baf64bafb3e838 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:12.701929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:12.703590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:12.720811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d89c9-ba09-4aa0-a3d5-688476011c7b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:12.780166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:12.781094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.781382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.781666 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] PATH_INFO: `/v3/projects/568f001ca8f74667acafd8f9e44fe68d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.782856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568f001ca8f74667acafd8f9e44fe68d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:12.784761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:12.797137 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:12.833837 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568f001ca8f74667acafd8f9e44fe68d', 'request_id': 'req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.839356 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568f001ca8f74667acafd8f9e44fe68d', 'request_id': 'req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.847105 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568f001ca8f74667acafd8f9e44fe68d', 'request_id': 'req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.858589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] The token cache is being invalidate because project 568f001ca8f74667acafd8f9e44fe68d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:12.858879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.859506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b78d15d3-14ff-43e3-80b3-c8f4c7ecaaf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:12.861063 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1386/2738] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:12 2022] DELETE /identity/v3/projects/568f001ca8f74667acafd8f9e44fe68d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:12.866079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.866607 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.866707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.884490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:12.885717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:12.943890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1353/2739] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:12.951516 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.951832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.952115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:12.969828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:12.971020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:12.978847 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1387/2740] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:35:12.983750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:12.993453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f905-d580-42db-ae45-ffe5a2e2b8c8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:12.997978 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:12.999264 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:12.999551 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:12.999832 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.000570 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:13.000898 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.002500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.014222 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1354/2741] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:13.018919 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.028119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29dcb2-e341-4a15-918f-11e8d1ca8c21 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.033456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.034120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.034410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.034701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.035411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.037429 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.066222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1388/2742] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:13 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:35:13.070323 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.080689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f275914-f80c-4ee2-9f3b-19181bbb6737 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.105122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.107061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:13.143274 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.143951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.144236 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.144512 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.145263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.147183 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.167374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1355/2743] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:13 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:13.172002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.179964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d245bf-0e46-4f22-b54a-c8a4b7b642c7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.181806 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fceb35-9996-4784-a90c-9cfa78794f14 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.185189 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.185881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.186319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.186610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.186902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.187561 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.187853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.188135 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.188399 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:13.188865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.190609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.196470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:13.201077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.202973 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.204707 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1356/2744] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:13.207611 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.214389 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1389/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:35:13.218344 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f941ef0-a587-48f2-b06f-c81e3293c1a5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.222965 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.223629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.223919 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.224203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.225022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.226620 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd4784-59c5-4247-b8cf-2d939439e05a admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.237702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1357/2746] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:13.241840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.252984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86262f-860d-44bb-b227-ee402632c31d service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.257596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.258483 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.258771 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.259052 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] PATH_INFO: `/v3/projects/841754b7d1914bcfa76aef0822c111ea/users/9c053c25e4ad41cd9c398b7b1f44596d/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.274986 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841754b7d1914bcfa76aef0822c111ea, user_id=9c053c25e4ad41cd9c398b7b1f44596d, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.279155 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.299802 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1390/2747] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:13 2022] PUT /identity/v3/projects/841754b7d1914bcfa76aef0822c111ea/users/9c053c25e4ad41cd9c398b7b1f44596d/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:13.305579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0970c2-73a0-4c4f-9090-60c42afa166b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.305874 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0970c2-73a0-4c4f-9090-60c42afa166b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.306161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0970c2-73a0-4c4f-9090-60c42afa166b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.332654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9e0970c2-73a0-4c4f-9090-60c42afa166b None None] MFA Rules not processed for user `9c053c25e4ad41cd9c398b7b1f44596d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:13.333909 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9e0970c2-73a0-4c4f-9090-60c42afa166b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:13.400499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1358/2748] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:13.411651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:13.413375 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:13.453848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:13.471331 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba03f-a0f5-4deb-bce2-3de546c97128 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:13.475003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:13.475651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:13.475936 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:13.476214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:13.484602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:13.486070 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f0bbe-8368-4b71-86f9-152d74ca321b service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:13.494885 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1391/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:14.832645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:14.833199 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:14.833296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:14.850555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] MFA Rules not processed for user `9c053c25e4ad41cd9c398b7b1f44596d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:14.851749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:14.858738 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1359/2750] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:23.490799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=9 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:24.845453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=40 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:28.420866 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97362-01d0-4c2e-bc6a-a7204d3f6d3f None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.421353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97362-01d0-4c2e-bc6a-a7204d3f6d3f None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.421450 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f97362-01d0-4c2e-bc6a-a7204d3f6d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.440079 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-18f97362-01d0-4c2e-bc6a-a7204d3f6d3f None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:28.441325 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-18f97362-01d0-4c2e-bc6a-a7204d3f6d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.496515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1392/2751] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:28.543120 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:28.544635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:28.613223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6fcda-e682-4303-a389-58f52352316d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:28.618240 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:28.618907 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:28.619200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:28.619488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:28.628233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:28.629866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:28.637952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1360/2752] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:35:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:35:29.170722 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb95a4-006d-4598-a215-1bf0e68310ed None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.171082 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb95a4-006d-4598-a215-1bf0e68310ed None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.171367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb95a4-006d-4598-a215-1bf0e68310ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.189020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c2fb95a4-006d-4598-a215-1bf0e68310ed None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:29.190212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c2fb95a4-006d-4598-a215-1bf0e68310ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.198675 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1393/2753] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:29.211165 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.222412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6da16-ce76-4e37-b2c5-2b015d150859 service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.227803 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.228444 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.228764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.229078 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.237205 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.238779 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.246706 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1361/2754] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:35:29.432550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77662-3c7c-40ce-b3d7-49d9d0b7d562 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.433032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77662-3c7c-40ce-b3d7-49d9d0b7d562 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.433212 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77662-3c7c-40ce-b3d7-49d9d0b7d562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.451015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c1e77662-3c7c-40ce-b3d7-49d9d0b7d562 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:29.452705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c1e77662-3c7c-40ce-b3d7-49d9d0b7d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.459697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1394/2755] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:29.471941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.482309 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab39d7-a9e2-4d19-bc81-ac9cac493ede service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.487194 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.487826 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.488115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.488391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.496397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.498012 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.506749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1362/2756] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 00:35:29.828711 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.829056 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.829347 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.847165 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] MFA Rules not processed for user `d58b01886c5d441da40130bffc817da7`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:29.848340 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:29.855562 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1395/2757] 10.222.0.23 () {58 vars in 967 bytes} [Thu Jun 16 00:35:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:29.867800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:29.879352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e245693-f532-4049-95bb-c0119ed920d3 service swift] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:29.884474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e080ca56d4f28bb1692d147cf4caa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:29.885146 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:29.885445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:29.885698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:29.894070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:29.895634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942e50d-74f1-4e77-9d2e-6fcb5344a16b service swift] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:29.903491 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1363/2758] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Jun 16 00:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 00:35:31.974582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.043901 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c4e5f-440a-4726-94cf-efe171d48326 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.048992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.049890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.050173 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.050451 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] PATH_INFO: `/v3/users/9c053c25e4ad41cd9c398b7b1f44596d` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.051628 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c053c25e4ad41cd9c398b7b1f44596d)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.053570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.099561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c053c25e4ad41cd9c398b7b1f44596d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.106614 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c053c25e4ad41cd9c398b7b1f44596d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.114244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c053c25e4ad41cd9c398b7b1f44596d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.114929 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c053c25e4ad41cd9c398b7b1f44596d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.126679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c053c25e4ad41cd9c398b7b1f44596d'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.132420 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1396/2759] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:35:31 2022] DELETE /identity/v3/users/9c053c25e4ad41cd9c398b7b1f44596d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:32.137160 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.137623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.137720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.156427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:32.157661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.225729 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1364/2760] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:32.235453 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.237071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.243641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:32.298648 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70fefa-904e-4b8c-b9a7-67cf4a631dce admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.302383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.303088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.303376 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.303656 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.313051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.314517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33670436-d3ed-4c5c-851d-e15346ee288d service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.324686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1397/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:32.497290 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.499371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.511878 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac6468-6d92-4657-b9b1-a3baf533ef63 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.517379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.518918 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.519675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.520006 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] PATH_INFO: `/v3/projects/841754b7d1914bcfa76aef0822c111ea` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.521427 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841754b7d1914bcfa76aef0822c111ea)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.523404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.570720 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:32.574134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841754b7d1914bcfa76aef0822c111ea', 'request_id': 'req-6fbf7c18-367a-49d5-b457-cb962aeb7295'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.581852 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841754b7d1914bcfa76aef0822c111ea', 'request_id': 'req-6fbf7c18-367a-49d5-b457-cb962aeb7295'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.587745 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841754b7d1914bcfa76aef0822c111ea', 'request_id': 'req-6fbf7c18-367a-49d5-b457-cb962aeb7295'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.601339 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] The token cache is being invalidate because project 841754b7d1914bcfa76aef0822c111ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:35:32.601630 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.602252 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-6fbf7c18-367a-49d5-b457-cb962aeb7295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:35:32.603795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1365/2762] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:35:32 2022] DELETE /identity/v3/projects/841754b7d1914bcfa76aef0822c111ea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:35:32.609051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.609349 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.609633 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.627958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:32.629187 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.690965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1398/2763] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:32.699220 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae938e6-3784-477c-a8df-06451865293d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.699511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae938e6-3784-477c-a8df-06451865293d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.699792 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae938e6-3784-477c-a8df-06451865293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.719404 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8ae938e6-3784-477c-a8df-06451865293d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:32.720616 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8ae938e6-3784-477c-a8df-06451865293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:32.728543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1366/2764] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:32.733275 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.743521 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f6d35-ebaf-4f7b-8b11-8410edf63060 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.748490 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.749672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.749957 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.750252 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.751047 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:35:32.751388 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.753184 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.766142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1399/2765] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:35:32.771265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae938e6-3784-477c-a8df-06451865293d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.781097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae938e6-3784-477c-a8df-06451865293d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.785761 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.786459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.786771 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.787084 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.787905 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.790046 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.817088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1367/2766] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:32.821262 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.831669 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15157728-f879-4a2f-b672-2949e16f70d7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.886459 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.887122 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.887411 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.887688 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.888367 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.890671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.910636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1400/2767] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:35:32 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:35:32.914810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.927619 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a6f8b-17bb-4d02-9e9e-d21b44b1d25d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.932168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.932830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.933152 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.933431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.934161 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.936347 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.948086 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1368/2768] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:35:32.952073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.962031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5162d-705b-4a9b-896d-a38ac1e96cc3 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:32.966653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:32.967407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:32.968320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:32.968320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:32.968768 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:32.970284 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183935e9-65a7-4785-93b1-52004d65c617 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:32.980555 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1401/2769] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:35:32.985824 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:32.996722 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5d9e2-d7cc-473e-bd04-446c296cf869 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:33.002285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:33.003276 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.003566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.003842 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] PATH_INFO: `/v3/projects/8dd0505146da40efbb028199390f8c75/users/f77cfd6c2be6487e9701e08340c6c792/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.020455 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd0505146da40efbb028199390f8c75, user_id=f77cfd6c2be6487e9701e08340c6c792, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:33.023216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:33.037097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1369/2770] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:35:32 2022] PUT /identity/v3/projects/8dd0505146da40efbb028199390f8c75/users/f77cfd6c2be6487e9701e08340c6c792/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:35:33.042561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ada6c-a4bf-4623-b774-2385fa8ca233 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.043046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ada6c-a4bf-4623-b774-2385fa8ca233 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.043323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ada6c-a4bf-4623-b774-2385fa8ca233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.063404 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-529ada6c-a4bf-4623-b774-2385fa8ca233 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:33.064624 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-529ada6c-a4bf-4623-b774-2385fa8ca233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:33.074163 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:33.131765 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1402/2771] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:33.142020 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:33.143889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:33.203995 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006963fc-efc9-4b7d-957d-5ab094348f35 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:33.208284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:33.209094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.209390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.210175 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.221663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:33.223108 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:33.233696 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1370/2772] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:33.849579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c700b5-6404-4ed9-828a-e9aff76da667 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:33.849908 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c700b5-6404-4ed9-828a-e9aff76da667 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:33.850246 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c700b5-6404-4ed9-828a-e9aff76da667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:33.877233 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-a6c700b5-6404-4ed9-828a-e9aff76da667 None None] MFA Rules not processed for user `f77cfd6c2be6487e9701e08340c6c792`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:33.878454 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-a6c700b5-6404-4ed9-828a-e9aff76da667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:33.950383 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1403/2773] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:33.961499 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:35:33.962915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:34.020968 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22409043-b45f-4b5d-90b9-11e87a0f0464 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:35:34.025386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:35:34.026105 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:34.026395 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:34.026673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:34.035672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:35:34.037131 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32795fac-a37b-4e2e-9564-13e2fffda7a8 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:35:34.046367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1371/2774] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:35:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:35:35.387268 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcd380-3b89-4669-8349-bfcdff412d94 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:35.387745 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcd380-3b89-4669-8349-bfcdff412d94 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:35.387853 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcd380-3b89-4669-8349-bfcdff412d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:35.406387 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7ddcd380-3b89-4669-8349-bfcdff412d94 None None] MFA Rules not processed for user `f77cfd6c2be6487e9701e08340c6c792`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:35.407582 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7ddcd380-3b89-4669-8349-bfcdff412d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:35.414407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1404/2775] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:43.000906 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6af8-e040-4906-9d47-667d740f3190 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:35:43.001440 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6af8-e040-4906-9d47-667d740f3190 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:35:43.001578 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6af8-e040-4906-9d47-667d740f3190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:35:43.020571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-133c6af8-e040-4906-9d47-667d740f3190 None None] MFA Rules not processed for user `d5985474b0774d8ea0e62718de767310`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:35:43.021935 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-133c6af8-e040-4906-9d47-667d740f3190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:35:43.032955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:43.083511 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1372/2776] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:35:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:35:45.400517 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=56 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:35:53.075785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=18 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:06.212171 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:06.212672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:06.212777 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:06.230700 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] MFA Rules not processed for user `728f4031df4148fe9ec9bf9ab31d25e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:06.232073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:06.290671 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1405/2777] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:36:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:16.283898 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=25 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:36:36.507114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:36:36.507673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:36:36.507673 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:36:36.525681 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] MFA Rules not processed for user `f77cfd6c2be6487e9701e08340c6c792`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:36:36.527030 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:36:36.534756 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1373/2778] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:36:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:36:46.520692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:02.226947 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:02.228470 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:02.241092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47866c0-fa13-4f48-b6a9-49be112dddc5 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:02.244587 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:02.245285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:02.245601 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:02.245899 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:02.247630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:02.317638 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:02.319142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:02.330297 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1406/2779] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:04.034545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:04.044200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6d8ae-b68c-4fef-88a6-c48b0e232e7b None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:04.048039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:04.048692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:04.049019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:04.049306 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:04.051081 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:04.065453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:04.067098 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:04.078456 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1374/2780] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:12.325302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=34 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:14.073314 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:15.229958 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:15.231508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:15.245918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd676063-16c2-4d93-af75-ff4bfc8ba40b service cinder] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:15.251315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:15.252222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.252513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.252790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] PATH_INFO: `/v3/users/d5985474b0774d8ea0e62718de767310` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.254016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5985474b0774d8ea0e62718de767310)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:15.255928 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:15.305422 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5985474b0774d8ea0e62718de767310'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.313574 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5985474b0774d8ea0e62718de767310'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.320714 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5985474b0774d8ea0e62718de767310'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.321457 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5985474b0774d8ea0e62718de767310'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.334729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e6e2adcf-2389-44b1-a63d-8746aaf05a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5985474b0774d8ea0e62718de767310'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.340692 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1407/2781] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:15 2022] DELETE /identity/v3/users/d5985474b0774d8ea0e62718de767310 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:15.539967 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:15.552488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eee748-2444-4e28-a7ed-7746aab2dde1 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:15.610204 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:15.611080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.611370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.611683 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] PATH_INFO: `/v3/projects/d2cc2e1e741d4eeb849e07d9f2c371be` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.612893 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cc2e1e741d4eeb849e07d9f2c371be)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:15.614840 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:15.661342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cc2e1e741d4eeb849e07d9f2c371be', 'request_id': 'req-cec721df-d987-44d9-9971-3011597b326b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.668102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cc2e1e741d4eeb849e07d9f2c371be', 'request_id': 'req-cec721df-d987-44d9-9971-3011597b326b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.674843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cc2e1e741d4eeb849e07d9f2c371be', 'request_id': 'req-cec721df-d987-44d9-9971-3011597b326b'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.688747 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] The token cache is being invalidate because project d2cc2e1e741d4eeb849e07d9f2c371be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:15.689221 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.689719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-cec721df-d987-44d9-9971-3011597b326b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:15.691435 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1375/2782] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:15 2022] DELETE /identity/v3/projects/d2cc2e1e741d4eeb849e07d9f2c371be => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:15.696855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.697218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.697568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.717337 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:15.718653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:15.791935 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1408/2783] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:15.800379 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.800674 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.801007 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.819627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:15.820865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:15.829104 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1376/2784] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:15.833895 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:15.844781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0a701-6126-40ff-b132-3e7a0e479fa2 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:15.849859 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:15.850533 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.850867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.851259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.852164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:15.852485 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:15.854733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:15.868782 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1409/2785] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:15.872979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:15.882533 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1eef0-42c1-46bc-812c-4c13feb9997e None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:15.887307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:15.888253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:15.888543 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:15.888827 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:15.889736 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:15.891686 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:15.921443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1377/2786] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:15.923096 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:15.934125 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e11a2-a079-4922-b77a-56016d633fc9 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.003116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.003841 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.004132 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.004417 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.005177 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.007127 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.014544 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:16.034881 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1410/2787] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:15 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:16.038445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.050198 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241169-8fa2-4ed1-a2f9-8cf3afe42d81 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.056168 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.056849 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.057173 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.057459 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.058201 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.059810 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.072899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1378/2788] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:16.076992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.087357 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2661b-cfd6-4af8-9d28-33cb7291a25f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.092147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.093600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.093949 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.094239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.095061 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.096662 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08c73a-0392-4ea7-84bb-05e46cadeef0 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.107618 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1411/2789] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:16.112411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.123277 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11e50b-2c15-4b38-aa80-94060c46166f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.128138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.129060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.129352 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.129633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] PATH_INFO: `/v3/projects/d8cc73c7367441769406ab11e9d308a6/users/c830b04dc5c34067bd49a94b99d5c259/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.144629 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cc73c7367441769406ab11e9d308a6, user_id=c830b04dc5c34067bd49a94b99d5c259, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.147477 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.161208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1379/2790] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:16 2022] PUT /identity/v3/projects/d8cc73c7367441769406ab11e9d308a6/users/c830b04dc5c34067bd49a94b99d5c259/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:16.172484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.172776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.173085 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.199232 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] MFA Rules not processed for user `c830b04dc5c34067bd49a94b99d5c259`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:16.200433 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:16.265022 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1412/2791] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:16.275986 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.277709 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:16.310171 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:16.354961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13dab9e-a4a6-4ade-bbdd-fd1aefd6c68b admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.360571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.361254 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.361547 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.361830 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.369684 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.371316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e7400-3a73-48ae-996a-17798cb7d29f service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.380334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1380/2792] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:16.597392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.660401 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b5fe9-f7d0-4714-bd10-5582c52aad47 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.664836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.665751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.666150 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.666316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] PATH_INFO: `/v3/users/c830b04dc5c34067bd49a94b99d5c259` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.667535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c830b04dc5c34067bd49a94b99d5c259)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.669476 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.715882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c830b04dc5c34067bd49a94b99d5c259'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:16.723806 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c830b04dc5c34067bd49a94b99d5c259'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:16.730815 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c830b04dc5c34067bd49a94b99d5c259'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:16.731519 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c830b04dc5c34067bd49a94b99d5c259'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:16.737334 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:16.745832 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c830b04dc5c34067bd49a94b99d5c259'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:16.756185 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1413/2793] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:16 2022] DELETE /identity/v3/users/c830b04dc5c34067bd49a94b99d5c259 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:16.760740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.761265 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.761386 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.779208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:16.780392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:16.841340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1381/2794] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:16.852170 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:16.853731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:16.922032 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d75e1-73f1-431b-9a20-dffc697ff941 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:16.925758 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:16.926550 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:16.926867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:16.927174 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:16.936719 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:16.938203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660c26c-47d9-4e5f-a869-a20c2806143e service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:16.948312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1414/2795] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:17.383979 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.385445 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:17.398408 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ae5d7-2b1d-4789-b802-f24649761ca4 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.403066 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.403939 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.404224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.404498 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] PATH_INFO: `/v3/projects/d8cc73c7367441769406ab11e9d308a6` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.415158 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cc73c7367441769406ab11e9d308a6)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.417226 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.471960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cc73c7367441769406ab11e9d308a6', 'request_id': 'req-0ff40d39-729b-447d-aa4d-02fee3feb7b8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:17.480185 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cc73c7367441769406ab11e9d308a6', 'request_id': 'req-0ff40d39-729b-447d-aa4d-02fee3feb7b8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:17.487652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cc73c7367441769406ab11e9d308a6', 'request_id': 'req-0ff40d39-729b-447d-aa4d-02fee3feb7b8'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:17.501334 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] The token cache is being invalidate because project d8cc73c7367441769406ab11e9d308a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86548) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:17.501633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:17.502263 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-0ff40d39-729b-447d-aa4d-02fee3feb7b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:17.504413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1382/2796] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:17 2022] DELETE /identity/v3/projects/d8cc73c7367441769406ab11e9d308a6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:17.509502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.510025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.510025 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.532652 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:17.533991 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:17.593217 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1415/2797] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:17.601340 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.601635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.601922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.620071 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:17.621296 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:17.629869 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1383/2798] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:17.634110 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.644843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f92a5-8005-4148-be44-8978eac2b011 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.649592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.650244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.650528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.650805 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.651527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86549) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:17.651834 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.653449 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.664831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1416/2799] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:17.669255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.679059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561dbe-51d3-46b2-8e85-f7f318862bd5 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.683743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.684392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.684680 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.684993 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.685702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.688568 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.716230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1384/2800] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:17 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:17.720419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.730729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b70b4-d045-4ccd-ab7c-b10d0874f6d8 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.786055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.786727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.787018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.787301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.788004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.789966 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.810073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1417/2801] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:17 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:17.814739 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.825059 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af2bf8-85af-4c53-9212-631b2cac2c01 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.830129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.830795 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.831080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.831362 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.832097 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.833737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.835039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:17.846526 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1385/2802] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:17.850226 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.860534 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc4371-f3aa-4f25-b303-6223020362e6 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.865109 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.865763 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.866049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.866328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.867049 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.868630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3cad0-f41b-4d01-8bdd-88a39cd1801c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.869865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:17.881641 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1418/2803] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:17.885410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:17.896042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aba289-cfff-418e-9c9a-8a3f57674739 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:17.900555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:17.901470 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.901754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.902029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] PATH_INFO: `/v3/projects/2a5543094f9f472ea824dab9fe2277b8/users/677d1b1b83994ae6940cb9eed7311e1a/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.916705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5543094f9f472ea824dab9fe2277b8, user_id=677d1b1b83994ae6940cb9eed7311e1a, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:17.919348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:17.932273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1386/2804] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:17 2022] PUT /identity/v3/projects/2a5543094f9f472ea824dab9fe2277b8/users/677d1b1b83994ae6940cb9eed7311e1a/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:17.942639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:17.942937 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:17.943219 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:17.969296 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] MFA Rules not processed for user `677d1b1b83994ae6940cb9eed7311e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:17.970518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:18.033239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1419/2805] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:18.037654 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.099176 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727ad8f-4130-4418-82d4-4c11bc3d7230 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.103694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.104354 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.104645 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.104955 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.105668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.107566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.133332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1387/2806] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:18 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:18.137319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.148386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b178ccf-0933-4e6a-bc76-03b2f4e608cd None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.202921 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.203593 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.203884 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.204164 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] PATH_INFO: `/v3/users` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.204861 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.206814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.229835 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1420/2807] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:18 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:18.231019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.243073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4393aa-345a-4b39-a68f-78a645b78d82 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.247794 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.248914 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.249454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.249984 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.251403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.253139 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.265836 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1388/2808] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:37:18.270090 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.280979 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eab60-c779-42cc-935d-1d5900b55f92 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.285797 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.286712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.287004 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.287288 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] PATH_INFO: `/v3/projects/e56c1672eafd4af68379a196a74fe4c0/users/1a2ccd3dcfc844a998db90a32eabbdfa/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.307016 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56c1672eafd4af68379a196a74fe4c0, user_id=1a2ccd3dcfc844a998db90a32eabbdfa, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.309744 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.323526 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1421/2809] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:18 2022] PUT /identity/v3/projects/e56c1672eafd4af68379a196a74fe4c0/users/1a2ccd3dcfc844a998db90a32eabbdfa/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:18.327848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.338293 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e19af-36d8-4a7e-a39e-cb361e2bf5a7 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.396479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.397192 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.397488 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.397768 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.398506 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.400102 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaaa097-f81c-4efe-b76f-3a96117cfb85 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.410661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1389/2810] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:18.414831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.426910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18022921-23c6-4bd2-a108-13b80b8a4f71 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.431980 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.432938 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.433231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.433512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] PATH_INFO: `/v3/projects/e56c1672eafd4af68379a196a74fe4c0/users/1a2ccd3dcfc844a998db90a32eabbdfa/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.435507 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56c1672eafd4af68379a196a74fe4c0, user_id=1a2ccd3dcfc844a998db90a32eabbdfa, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.438848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.450152 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1422/2811] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:18 2022] PUT /identity/v3/projects/e56c1672eafd4af68379a196a74fe4c0/users/1a2ccd3dcfc844a998db90a32eabbdfa/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:18.462178 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.462541 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.463045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.491312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] MFA Rules not processed for user `1a2ccd3dcfc844a998db90a32eabbdfa`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:18.492522 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:18.534138 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:18.581797 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1390/2812] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:18.595992 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:18.597778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:18.642439 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:18.658528 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece9eb0-6546-4a39-b51a-86441ae7e75f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:18.662113 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:18.662761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:18.663051 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:18.663336 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:18.674197 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:18.675639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7402af7-5d83-4bec-ad95-fbbb6be6c19e service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:18.685188 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1423/2813] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:19.083039 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:19.145303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0adfe-5ea1-4435-95c3-37b07b6dd9cf None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:19.149855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:19.150732 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.151024 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.151308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] PATH_INFO: `/v3/users/677d1b1b83994ae6940cb9eed7311e1a` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.152508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677d1b1b83994ae6940cb9eed7311e1a)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:19.154460 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:19.199802 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677d1b1b83994ae6940cb9eed7311e1a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.207529 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677d1b1b83994ae6940cb9eed7311e1a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.220055 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677d1b1b83994ae6940cb9eed7311e1a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.220698 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677d1b1b83994ae6940cb9eed7311e1a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.221327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677d1b1b83994ae6940cb9eed7311e1a'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.234099 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1391/2814] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:19 2022] DELETE /identity/v3/users/677d1b1b83994ae6940cb9eed7311e1a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:19.238687 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.239059 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.239418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.256860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:19.258093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:19.317323 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1424/2815] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:19.326833 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:19.328300 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:19.380203 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:19.386462 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bd4d4-d1e9-4822-9448-98e5f4d32a75 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:19.391413 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:19.392073 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.392366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.392687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.401814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:19.403253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:19.413320 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1392/2816] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:19.692510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:19.694748 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:19.708731 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb864ff-0116-4d2a-812f-f758b38e9fce None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:19.713931 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:19.715045 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.715505 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.715788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] PATH_INFO: `/v3/projects/2a5543094f9f472ea824dab9fe2277b8` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.717172 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5543094f9f472ea824dab9fe2277b8)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:19.719594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:19.782057 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5543094f9f472ea824dab9fe2277b8', 'request_id': 'req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.787890 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5543094f9f472ea824dab9fe2277b8', 'request_id': 'req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.795106 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5543094f9f472ea824dab9fe2277b8', 'request_id': 'req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.807324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] The token cache is being invalidate because project 2a5543094f9f472ea824dab9fe2277b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:19.807621 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.808276 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.809879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1425/2817] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:19 2022] DELETE /identity/v3/projects/2a5543094f9f472ea824dab9fe2277b8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:19.813912 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:19.823414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab27a8-ff49-4b16-ad18-62defac6c135 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:19.879560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:19.880429 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:19.880737 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:19.881038 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] PATH_INFO: `/v3/users/1a2ccd3dcfc844a998db90a32eabbdfa` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:19.882228 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2ccd3dcfc844a998db90a32eabbdfa)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:19.884141 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:19.932111 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2ccd3dcfc844a998db90a32eabbdfa'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.939372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2ccd3dcfc844a998db90a32eabbdfa'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.951281 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2ccd3dcfc844a998db90a32eabbdfa'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.952565 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2ccd3dcfc844a998db90a32eabbdfa'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.953231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f1a0df5a-d6e5-4a5a-ad8f-54fe89831804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2ccd3dcfc844a998db90a32eabbdfa'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:19.965235 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1393/2818] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:19 2022] DELETE /identity/v3/users/1a2ccd3dcfc844a998db90a32eabbdfa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:20.235050 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.236460 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:20.249480 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c43dc-81e9-4d3e-938c-8b6492eed86c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.307213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.308074 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.308360 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.308654 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] PATH_INFO: `/v3/projects/e56c1672eafd4af68379a196a74fe4c0` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.309856 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56c1672eafd4af68379a196a74fe4c0)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.312015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.318038 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:20.359229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56c1672eafd4af68379a196a74fe4c0', 'request_id': 'req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.364535 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56c1672eafd4af68379a196a74fe4c0', 'request_id': 'req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.376018 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56c1672eafd4af68379a196a74fe4c0', 'request_id': 'req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.387041 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] The token cache is being invalidate because project e56c1672eafd4af68379a196a74fe4c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:20.387330 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.388218 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-b6ca705c-9d5f-46b6-9d56-2c5e275262c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:20.390984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1426/2819] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:20 2022] DELETE /identity/v3/projects/e56c1672eafd4af68379a196a74fe4c0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:20.395964 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.396421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.396548 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.414184 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:20.415374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:20.475249 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1394/2820] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:20.483093 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.483385 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.483685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.502123 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:20.503240 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:20.510964 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1427/2821] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:20.516021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.525672 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847800b-cf86-42f5-93c3-8d71a80f92b9 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.530156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.530809 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.531095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.531372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.532094 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:20.532403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.534022 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.536327 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:20.547223 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1395/2822] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:37:20.551270 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.560697 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d6508-dbb2-49fc-9b4b-6fa2ec11a7db None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.565819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.566469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.566755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.567033 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.567712 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.569620 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.596390 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1428/2823] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:20.600397 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.610569 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d2b12-5ae5-4e61-9739-c3d5c4a1c05c admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.666819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.667487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.667777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.668060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.668758 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.670838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.691269 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1396/2824] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:20.695418 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.706558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d69b-b287-46f3-b987-a1d7aae20082 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.711210 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.711879 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.712166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.712447 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.713214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.714813 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.725400 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1429/2825] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:20.729454 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.740045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488c58-cd46-4486-8704-6d629bfd1ca7 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.744704 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.745401 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.745689 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.745969 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.746697 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.748315 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64016ee3-fd41-4d97-9848-a75875d5e664 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.759115 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1397/2826] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:37:20.763445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.773531 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9744e-628e-4813-923d-9870451d0d42 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.778324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.779209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.779492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.779803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] PATH_INFO: `/v3/projects/b7ca4a231c82404f9ad57464598b00f9/users/9510205a3ab34e2aa27fc68a8dcfc213/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.795139 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ca4a231c82404f9ad57464598b00f9, user_id=9510205a3ab34e2aa27fc68a8dcfc213, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.798518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:20.813394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1430/2827] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:20 2022] PUT /identity/v3/projects/b7ca4a231c82404f9ad57464598b00f9/users/9510205a3ab34e2aa27fc68a8dcfc213/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:20.823647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.823949 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.824238 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.850566 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] MFA Rules not processed for user `9510205a3ab34e2aa27fc68a8dcfc213`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:20.851764 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:20.914971 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1398/2828] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:20.919462 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:20.981243 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e5174-c4a9-4edf-8ae8-f7d469642b6b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:20.986838 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:20.987549 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:20.987842 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:20.988124 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:20.988823 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:20.990750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.016477 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1431/2829] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:20 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:21.020482 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.031780 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862cf3-4b5c-424c-8a22-2dec4da334ff None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.087744 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.088412 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.088699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.089011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.089699 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.091636 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.100555 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:21.113054 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1399/2830] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:21 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:21.117147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.127159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2781b27-e8df-4ca4-ad2c-a8862533d55c admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.132161 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.132820 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.133135 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.133416 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.134144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.135735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.146244 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1432/2831] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:21.150272 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.160599 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7ae4e-07fb-40fe-95b9-6c1832eea804 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.165224 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.166101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.166394 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.166679 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] PATH_INFO: `/v3/projects/9f99c282b3e947d783c159be8525ccdf/users/64af6315b761485f94f0b412bd7056e0/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.182193 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f99c282b3e947d783c159be8525ccdf, user_id=64af6315b761485f94f0b412bd7056e0, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.184841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.197743 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1400/2832] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:21 2022] PUT /identity/v3/projects/9f99c282b3e947d783c159be8525ccdf/users/64af6315b761485f94f0b412bd7056e0/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:21.201740 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.208094 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:21.213778 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e245f-9307-40e4-8c60-c99aa614a2e9 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.273705 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.274301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.274590 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.274869 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.275597 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.277225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f5653-090e-47af-af1f-5b8be2b753c7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.288520 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1433/2833] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:21.292749 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.304230 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e13fc4-3e2d-4c62-94de-ef5df2602ae5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.308801 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.309718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.310004 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.310286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] PATH_INFO: `/v3/projects/9f99c282b3e947d783c159be8525ccdf/users/64af6315b761485f94f0b412bd7056e0/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.312718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f99c282b3e947d783c159be8525ccdf, user_id=64af6315b761485f94f0b412bd7056e0, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.315391 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.325651 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1401/2834] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:21 2022] PUT /identity/v3/projects/9f99c282b3e947d783c159be8525ccdf/users/64af6315b761485f94f0b412bd7056e0/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:21.337407 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.337701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.337984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.364703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] MFA Rules not processed for user `64af6315b761485f94f0b412bd7056e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:21.365941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:21.433506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1434/2835] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:21 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:21.445270 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.446962 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:21.503940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964531d-26d4-4109-b53c-4e003e13adda admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.507515 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.508149 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.508433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.508711 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.518255 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.519708 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.529061 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1402/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:21.607996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:21.617572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2e941-cb53-41cf-8e82-06a89cc8a1ec None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:21.621142 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:21.621776 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:21.622066 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:21.622342 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:21.680415 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:21.681880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6914b1-8cf3-411d-a71f-85ef75cba41d service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:21.691501 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1435/2837] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:26.332841 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:26.395253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8287ac-5f61-4789-842b-dc5b871bd470 service cinder] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:26.399719 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:26.400591 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:26.400901 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:26.401186 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] PATH_INFO: `/v3/users/9510205a3ab34e2aa27fc68a8dcfc213` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:26.402371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9510205a3ab34e2aa27fc68a8dcfc213)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:26.404284 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:26.458372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9510205a3ab34e2aa27fc68a8dcfc213'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:26.466484 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9510205a3ab34e2aa27fc68a8dcfc213'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:26.472342 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:26.481008 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9510205a3ab34e2aa27fc68a8dcfc213'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:26.482010 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9510205a3ab34e2aa27fc68a8dcfc213'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:26.483243 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9510205a3ab34e2aa27fc68a8dcfc213'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:26.495372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1403/2838] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:26 2022] DELETE /identity/v3/users/9510205a3ab34e2aa27fc68a8dcfc213 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:26.500159 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:26.500639 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:26.500922 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:26.519223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:26.520741 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:26.542602 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=100 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:26.583091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1436/2839] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:26.593657 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:26.595151 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:26.655959 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9cf21-f0b2-42fd-a6ac-ba6750242cfe admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:26.659675 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:26.660326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:26.660613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:26.660897 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:26.671040 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:26.672661 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:26.682625 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1404/2840] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 00:37:26.957651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:26.959141 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:26.974761 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a46af-57fd-4e54-8bf5-a56e9e4e76ee None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:26.979635 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:26.980504 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:26.980867 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:26.981101 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] PATH_INFO: `/v3/projects/b7ca4a231c82404f9ad57464598b00f9` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:26.982302 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ca4a231c82404f9ad57464598b00f9)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:26.984540 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.031162 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ca4a231c82404f9ad57464598b00f9', 'request_id': 'req-bff56854-2989-4398-a3df-a5f159644580'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.037133 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ca4a231c82404f9ad57464598b00f9', 'request_id': 'req-bff56854-2989-4398-a3df-a5f159644580'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.044271 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ca4a231c82404f9ad57464598b00f9', 'request_id': 'req-bff56854-2989-4398-a3df-a5f159644580'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.055294 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] The token cache is being invalidate because project b7ca4a231c82404f9ad57464598b00f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:27.055591 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.056225 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.057814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1437/2841] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:26 2022] DELETE /identity/v3/projects/b7ca4a231c82404f9ad57464598b00f9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:27.061819 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.071411 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058174a-4e76-4aa3-b781-db94a1f7b077 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.127421 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.128299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.128589 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.128873 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] PATH_INFO: `/v3/users/64af6315b761485f94f0b412bd7056e0` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.130092 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64af6315b761485f94f0b412bd7056e0)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.132015 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.177231 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64af6315b761485f94f0b412bd7056e0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.184414 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64af6315b761485f94f0b412bd7056e0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.196992 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64af6315b761485f94f0b412bd7056e0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.197627 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64af6315b761485f94f0b412bd7056e0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.198232 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-ef9d06fd-75f2-4b5f-8b96-5f41182bbb56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64af6315b761485f94f0b412bd7056e0'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.207002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:27.211503 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1405/2842] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:27 2022] DELETE /identity/v3/users/64af6315b761485f94f0b412bd7056e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:27.476011 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.477510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:27.492442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff56854-2989-4398-a3df-a5f159644580 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.553672 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.554694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.555015 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.555299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] PATH_INFO: `/v3/projects/9f99c282b3e947d783c159be8525ccdf` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.556686 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f99c282b3e947d783c159be8525ccdf)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.558811 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.608013 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f99c282b3e947d783c159be8525ccdf', 'request_id': 'req-8039cdea-cd70-4b80-9ddc-d7668fc0e280'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.609430 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:27.614716 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f99c282b3e947d783c159be8525ccdf', 'request_id': 'req-8039cdea-cd70-4b80-9ddc-d7668fc0e280'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.624515 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f99c282b3e947d783c159be8525ccdf', 'request_id': 'req-8039cdea-cd70-4b80-9ddc-d7668fc0e280'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.635883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] The token cache is being invalidate because project 9f99c282b3e947d783c159be8525ccdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:27.636211 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.637115 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-8039cdea-cd70-4b80-9ddc-d7668fc0e280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:27.639229 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1438/2843] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:27 2022] DELETE /identity/v3/projects/9f99c282b3e947d783c159be8525ccdf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:27.645312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.645936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.645936 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.666493 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:27.668195 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:27.732500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1406/2844] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 00:37:27.740369 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.740660 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.740982 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.758637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:27.759831 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:27.768472 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1439/2845] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:27.772575 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.783011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f7c9b-6b80-4489-820f-ded5bb5d322d None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.787895 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.788558 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.788860 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.789153 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.789899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:27.790215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.791808 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.803875 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1407/2846] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 00:37:27.808314 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.818040 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf0c9-0e9d-480c-9a7a-a03d68b94443 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.823295 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.824027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.824316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.824595 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.825382 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.827926 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.854030 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1440/2847] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:27 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:27.858501 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.868407 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb70fd7-f589-4cf6-8f1e-4394e02582af admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.926360 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.927033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.927325 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.927609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.928316 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.930273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.952535 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1408/2848] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:27 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:27.956742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.967209 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70069d-2b7c-4c80-a55e-0bc07c6e0b42 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:27.971734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:27.972395 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:27.972682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:27.972996 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:27.973733 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:27.975319 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:27.985695 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1441/2849] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:27.989426 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:27.999838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7771da1-8ee3-41bf-bc26-1252feb9470b admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:28.004425 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:28.005118 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.005406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.005687 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.006417 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:28.008029 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768dfe3-d94d-4ed4-af86-552eb17eeb8b admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:28.018865 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1409/2850] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:28.022948 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:28.033910 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211c5a3-1a0c-4a7e-ae2d-cf53607d2687 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:28.038914 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:28.039788 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.040071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.040392 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] PATH_INFO: `/v3/projects/698d82b7eb624a57bae73527a533ada2/users/18f8a376acf94701a24b5d8f7c8cc8bf/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.055386 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=698d82b7eb624a57bae73527a533ada2, user_id=18f8a376acf94701a24b5d8f7c8cc8bf, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:28.058046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:28.071199 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1442/2851] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:28 2022] PUT /identity/v3/projects/698d82b7eb624a57bae73527a533ada2/users/18f8a376acf94701a24b5d8f7c8cc8bf/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:28.083997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.084307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.084571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.111207 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] MFA Rules not processed for user `18f8a376acf94701a24b5d8f7c8cc8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:28.112419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:28.171372 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:28.181239 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1410/2852] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:28.191953 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:28.193703 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:28.254183 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938cd3-9f97-4253-9257-18e2f75d0748 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:28.258167 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8863fd88c1a2416c9fdb0148db03f6e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:28.258827 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:28.259114 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:28.259394 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:28.268359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:28.269816 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a71e2-c565-4378-a34b-8a591dbd0e2a service cinder] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:28.278441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1443/2853] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:31.287823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:31.351508 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1665fc-4ecd-4e02-ba85-bc2b21deb56f None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:31.356633 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:31.358089 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:31.358388 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:31.358666 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] PATH_INFO: `/v3/users/18f8a376acf94701a24b5d8f7c8cc8bf` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:31.360014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f8a376acf94701a24b5d8f7c8cc8bf)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:31.362033 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:31.408604 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f8a376acf94701a24b5d8f7c8cc8bf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.415789 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f8a376acf94701a24b5d8f7c8cc8bf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.428214 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f8a376acf94701a24b5d8f7c8cc8bf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.428859 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f8a376acf94701a24b5d8f7c8cc8bf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.429505 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f8a376acf94701a24b5d8f7c8cc8bf'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.442267 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1411/2854] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:31 2022] DELETE /identity/v3/users/18f8a376acf94701a24b5d8f7c8cc8bf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:31.446846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:31.447335 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:31.447589 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:31.465134 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:31.466327 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:31.528091 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1444/2855] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:31.538487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:31.539900 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:31.601079 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f98e-482a-465e-a509-709ed10a0d1e admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:31.604718 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:31.605410 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:31.605702 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:31.605980 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:31.615930 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:31.617405 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095f2aa-d58f-43ce-b47c-c84cc90269e7 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:31.626850 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1412/2856] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:31.892320 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:31.893821 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:31.906883 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae296c6-224d-4176-b9e3-1607a9bb3eab None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:31.911568 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:31.912441 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:31.912729 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:31.913046 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] PATH_INFO: `/v3/projects/698d82b7eb624a57bae73527a533ada2` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:31.914238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=698d82b7eb624a57bae73527a533ada2)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:31.916158 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:31.929298 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:31.969381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698d82b7eb624a57bae73527a533ada2', 'request_id': 'req-e65aee34-ab2d-44d9-8bc9-c9153410d010'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.974848 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698d82b7eb624a57bae73527a533ada2', 'request_id': 'req-e65aee34-ab2d-44d9-8bc9-c9153410d010'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.981876 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698d82b7eb624a57bae73527a533ada2', 'request_id': 'req-e65aee34-ab2d-44d9-8bc9-c9153410d010'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.993203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] The token cache is being invalidate because project 698d82b7eb624a57bae73527a533ada2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:31.993495 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.994120 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e65aee34-ab2d-44d9-8bc9-c9153410d010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:31.995651 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1445/2857] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:31 2022] DELETE /identity/v3/projects/698d82b7eb624a57bae73527a533ada2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:32.000796 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.001208 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.001560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.019754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:32.021062 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:32.034036 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:32.083075 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1413/2858] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:32.090932 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.091223 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.091511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.109880 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:32.111076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:32.118717 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1446/2859] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:32.122652 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.132823 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561718b6-1c4c-4686-b12f-eb4bae19f538 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.137848 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.139125 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.139424 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.139798 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] PATH_INFO: `/v3/domains` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.140542 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86548) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 00:37:32.140867 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.142517 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.153899 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1414/2860] 10.222.0.23 () {58 vars in 1152 bytes} [Thu Jun 16 00:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 00:37:32.159186 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.168250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30201ad2-4b7f-4d4d-8249-55ad30ec3ab0 None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.173213 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.173863 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.174144 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.174419 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.175146 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.177073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.207118 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1447/2861] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:32 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 00:37:32.208579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.220253 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc99e-cf10-44be-8703-67e2320e339f admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.280940 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.281595 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.281951 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.282273 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.283217 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.285364 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.306641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1415/2862] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:32 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:32.310828 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.321317 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ddc87-dbb5-4f44-b6ba-82b97d95817e admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.326772 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.327473 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.327764 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.328055 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.328801 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.330475 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.341175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1448/2863] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 00:37:32.345889 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.356093 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083b7f2-ea5e-4713-9da4-cdd8376ceb3d admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.361378 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.362042 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.362333 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.362617 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] PATH_INFO: `/v3/roles` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.363381 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.364997 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17b754-0fd4-4b21-aa79-c59611b49198 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.376285 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1416/2864] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:32.380326 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.391097 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1f1ff-129d-438e-a8b3-216ccb3b6f3f admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.395698 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.396567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] REQUEST_METHOD: `PUT` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.396852 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.397156 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] PATH_INFO: `/v3/projects/e831286870b24b39bc1f72f087bf970e/users/218cff6f99a74a3f8e5cdd4e9787539d/roles/f5a6a04dca1e4234b7616969cfcd7643` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.412871 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e831286870b24b39bc1f72f087bf970e, user_id=218cff6f99a74a3f8e5cdd4e9787539d, role_id=f5a6a04dca1e4234b7616969cfcd7643)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.415600 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.428830 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1449/2865] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:32 2022] PUT /identity/v3/projects/e831286870b24b39bc1f72f087bf970e/users/218cff6f99a74a3f8e5cdd4e9787539d/roles/f5a6a04dca1e4234b7616969cfcd7643 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:32.433870 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.434262 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.434489 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.453366 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] MFA Rules not processed for user `bfaf1438c4804d7499ba789f509be85e`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:32.454855 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:32.516130 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1417/2866] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:32.525807 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:32.527573 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:32.586836 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06908d4-b47b-47ce-aa56-ae9d2b4a3238 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:32.590512 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:32.591166 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:32.591456 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:32.592285 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:32.601759 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:32.603198 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:32.612843 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1450/2867] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:33.919002 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:33.922592 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:33.942804 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b2047-597c-4883-9465-d3a8e057c7cd None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:33.954583 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:33.955550 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:33.955843 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:33.956128 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] PATH_INFO: `/v3/users/f77cfd6c2be6487e9701e08340c6c792` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:33.957957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77cfd6c2be6487e9701e08340c6c792)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:33.961011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:34.000635 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:34.036839 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77cfd6c2be6487e9701e08340c6c792'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.051609 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77cfd6c2be6487e9701e08340c6c792'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.066712 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77cfd6c2be6487e9701e08340c6c792'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.067582 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77cfd6c2be6487e9701e08340c6c792'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.068317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-f09dcd85-1332-4648-a65c-82abb1dd450e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77cfd6c2be6487e9701e08340c6c792'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.085785 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1418/2868] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:37:33 2022] DELETE /identity/v3/users/f77cfd6c2be6487e9701e08340c6c792 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:34.312583 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:34.324538 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f820f0-1c81-4f3d-9af1-c044afbb4daf service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:34.371510 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:34.393235 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:34.394299 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:34.394673 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:34.395003 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] PATH_INFO: `/v3/projects/8dd0505146da40efbb028199390f8c75` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:34.396247 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd0505146da40efbb028199390f8c75)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:34.398283 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:34.446263 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd0505146da40efbb028199390f8c75', 'request_id': 'req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.453071 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd0505146da40efbb028199390f8c75', 'request_id': 'req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.461943 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd0505146da40efbb028199390f8c75', 'request_id': 'req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.474353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] The token cache is being invalidate because project 8dd0505146da40efbb028199390f8c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:37:34.474645 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.475310 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:37:34.476873 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1451/2869] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:37:34 2022] DELETE /identity/v3/projects/8dd0505146da40efbb028199390f8c75 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 00:37:36.779156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:36.779671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:36.779671 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:36.848307 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] MFA Rules not processed for user `218cff6f99a74a3f8e5cdd4e9787539d`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:36.849915 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:36.938474 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1419/2870] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:36.942965 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:36.944445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:37.021027 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf0603-f5e9-4f92-a99b-86dc2e9415b7 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.027679 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.028438 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.029410 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.029723 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] PATH_INFO: `/v3/projects` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.030524 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.032445 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.061239 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1452/2871] 10.222.0.23 () {60 vars in 1152 bytes} [Thu Jun 16 00:37:36 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:37.065299 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.077101 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088a3fb-40f9-4680-bba7-941faf818fb3 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.136961 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.138245 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.138685 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.139241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.139956 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.141922 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.162200 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1420/2872] 10.222.0.23 () {60 vars in 1143 bytes} [Thu Jun 16 00:37:37 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 00:37:37.165775 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.176710 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024bd8-cb09-4b17-9379-157acd2e8915 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.182735 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.183406 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.183694 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.183971 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.184704 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.186381 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.197814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1453/2873] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:37.202047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.213998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771dd43-bc22-4087-88ee-a31f2dcd86b9 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.222317 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.223374 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.223667 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.223947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] PATH_INFO: `/v3/projects/d6fb3315274a42df8d139749b67d845b/users/1977cb9a5ab7492c8814b43eb471254c/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.237643 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:37.241241 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fb3315274a42df8d139749b67d845b, user_id=1977cb9a5ab7492c8814b43eb471254c, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.245019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.260148 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1421/2874] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:37 2022] PUT /identity/v3/projects/d6fb3315274a42df8d139749b67d845b/users/1977cb9a5ab7492c8814b43eb471254c/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:37.264933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.275316 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60d842-89d3-441c-b06f-b9ddcdb701ec admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.317292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:37.333814 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.334527 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.334819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.335102 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.335840 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.337486 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fd4ce-320d-40b0-9f80-12409dfcc2f8 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.347518 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1454/2875] 10.222.0.23 () {58 vars in 1121 bytes} [Thu Jun 16 00:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 00:37:37.351705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:37.362902 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e8bd7-8101-4103-bc83-dae8926c1ace admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:37.367472 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:37.368351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:37.368634 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:37.368938 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] PATH_INFO: `/v3/projects/d6fb3315274a42df8d139749b67d845b/users/1977cb9a5ab7492c8814b43eb471254c/roles/8e8886c4262d40709f5bdc535e1e89cb` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:37.370890 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fb3315274a42df8d139749b67d845b, user_id=1977cb9a5ab7492c8814b43eb471254c, role_id=8e8886c4262d40709f5bdc535e1e89cb)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:37.374348 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:37.384469 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1422/2876] 10.222.0.23 () {60 vars in 1482 bytes} [Thu Jun 16 00:37:37 2022] PUT /identity/v3/projects/d6fb3315274a42df8d139749b67d845b/users/1977cb9a5ab7492c8814b43eb471254c/roles/8e8886c4262d40709f5bdc535e1e89cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:37:41.999707 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c56cf-e640-44bc-969d-03457eab3bf5 None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:42.000353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c56cf-e640-44bc-969d-03457eab3bf5 None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:42.000353 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c56cf-e640-44bc-969d-03457eab3bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:42.030442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-c37c56cf-e640-44bc-969d-03457eab3bf5 None None] MFA Rules not processed for user `1977cb9a5ab7492c8814b43eb471254c`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:42.031682 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-c37c56cf-e640-44bc-969d-03457eab3bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:42.113637 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1455/2877] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:42.131676 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:37:42.133728 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:42.223513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e9f0-8a8a-4d7a-a35f-75d8be7fcfd1 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:37:42.231172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:37:42.231172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:42.231172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:42.231172 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:42.293403 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:37:42.295880 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:37:42.311393 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1423/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:37:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:37:43.335502 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] REQUEST_METHOD: `POST` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:37:43.336259 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:37:43.336428 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:37:43.355255 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.auth.core [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] MFA Rules not processed for user `218cff6f99a74a3f8e5cdd4e9787539d`. Rule list: `[]` (Enabled: `True`). {{(pid=86549) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:37:43.356630 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:37:43.364324 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1456/2879] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:37:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:37:52.301613 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:37:53.349558 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=46 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:00.335957 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:00.337487 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:00.409312 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eaf8d-f742-477d-b892-856797d603f1 service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:00.414359 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:00.415009 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:00.415297 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:00.415581 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:00.417380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:00.429303 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:00.430868 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:00.440825 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1424/2880] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:02.192594 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:02.194107 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:02.254617 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed992-770b-4a0c-a4bd-e2c474b9b04c None None] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:02.258264 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:02.258924 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:02.259214 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:02.259492 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:02.261256 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:02.271846 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:02.273301 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:02.282073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1457/2881] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:02.910947 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:02.912453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:02.929941 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27f6f5-8603-4364-9960-b3d6fef33a93 service placement] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:02.935663 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:02.936343 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:02.936647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:02.936976 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:02.938828 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:02.952406 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:02.954500 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:02.969717 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1425/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:12.277742 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=26 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:12.962677 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=38 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:15.057962 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:15.075116 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcafc9-9501-46c3-9d8c-d5a20b306ff2 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:15.082750 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:15.083570 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:15.083860 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:15.084145 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:15.085989 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:15.100902 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:15.102509 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:15.115154 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1458/2883] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:18.081080 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:18.082521 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:18.094481 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2af6d9-18d1-46e1-ba01-ddc3d798d0fb service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:18.098019 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:18.098655 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:18.098943 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:18.099225 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:18.108014 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:18.109464 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1eb53c-6043-416f-9b81-d421ac246b4e service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:18.118942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1426/2884] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:19.897685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:19.910930 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69b5-435d-48b2-8734-de4735e2aaf1 service nova] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:19.916825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:19.917514 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:19.917804 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:19.918088 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:19.919918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:19.933596 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:19.935398 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:19.946918 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1459/2885] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:24.686438 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] REQUEST_METHOD: `POST` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:24.686921 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:24.687045 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:24.704433 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.auth.core [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] MFA Rules not processed for user `728f4031df4148fe9ec9bf9ab31d25e5`. Rule list: `[]` (Enabled: `True`). {{(pid=86548) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 00:38:24.706443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:24.761507 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1427/2886] 10.222.0.23 () {58 vars in 957 bytes} [Thu Jun 16 00:38:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 00:38:25.068755 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:25.078031 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029700d8-690a-42b9-a390-8fce2e9dd349 service nova] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:25.081773 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:25.082413 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:25.082701 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:25.082985 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:25.084734 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:25.094403 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:25.095969 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:25.104001 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1460/2887] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:29.011814 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:38:29.024733 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368c03f-7508-4b59-bc76-5152c905f1a8 None None] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:38:29.030430 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50db388db1446eeb60ccaa17ff55389', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:38:29.031106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:38:29.031382 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:38:29.031668 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:38:29.034027 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:38:29.045181 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:38:29.046861 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:38:29.056788 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1428/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:38:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:38:35.100345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=18 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:38:39.052367 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=37 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:15.147855 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:15.222253 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7298b-4e9c-459b-95d2-350a1c6d1ca7 service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:15.229216 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:15.230623 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:15.231530 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:15.231819 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:15.234303 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:15.271751 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:15.273443 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:15.283799 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1461/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:25.041603 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:25.070479 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019743d-2dfb-46b9-8e3e-a71b1036625a service nova] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:25.074215 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:25.074853 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:25.075136 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:25.075431 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:25.077216 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:25.160248 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:25.161703 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:25.182370 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1429/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:25.279580 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=36 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:28.392561 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:28.394506 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:28.409312 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d442ef74-04da-4717-bbb2-aa2709f68c99 service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:28.412467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:28.413241 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:28.413592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:28.413941 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:28.415790 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:28.428464 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:28.430073 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:28.441341 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1462/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:31.668838 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.679124 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098b4fe-abcc-4785-9592-d8e3edf1075f service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.682640 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.683998 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.684291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.684571 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.686557 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:31.718942 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.720392 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.729952 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1430/2892] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:31.739886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.756175 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd088108-9042-4276-a684-a14b3eec1d0d service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.764854 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.765545 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.766020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.766471 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.769182 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:31.785825 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.787888 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.800988 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1463/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:31.820815 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:31.822436 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:31.844750 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7eedb-8596-4c79-a9d7-085aaa023763 service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:31.849847 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:31.850757 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:31.851047 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:31.851332 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] PATH_INFO: `/v3/users/728f4031df4148fe9ec9bf9ab31d25e5` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:31.852579 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=728f4031df4148fe9ec9bf9ab31d25e5)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:31.854596 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:31.907638 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728f4031df4148fe9ec9bf9ab31d25e5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.914923 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728f4031df4148fe9ec9bf9ab31d25e5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.927453 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728f4031df4148fe9ec9bf9ab31d25e5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.928114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728f4031df4148fe9ec9bf9ab31d25e5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.928730 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728f4031df4148fe9ec9bf9ab31d25e5'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:31.934970 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:31.943622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1431/2894] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:31 2022] DELETE /identity/v3/users/728f4031df4148fe9ec9bf9ab31d25e5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.113231 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.124955 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be14438-87f0-4e31-a6c6-dcbd2967f00e service placement] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.182414 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.183915 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.184683 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.185007 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] PATH_INFO: `/v3/projects/193571b962dc4fa1aac98a6acbdc9370` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.186205 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193571b962dc4fa1aac98a6acbdc9370)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.188128 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.233529 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193571b962dc4fa1aac98a6acbdc9370', 'request_id': 'req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.239147 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193571b962dc4fa1aac98a6acbdc9370', 'request_id': 'req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.246610 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193571b962dc4fa1aac98a6acbdc9370', 'request_id': 'req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.257412 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] The token cache is being invalidate because project 193571b962dc4fa1aac98a6acbdc9370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:32.257709 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.258344 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.259882 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1464/2895] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:32 2022] DELETE /identity/v3/projects/193571b962dc4fa1aac98a6acbdc9370 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.264377 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.265811 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:32.279113 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd58ed7-7619-4ee4-b53c-c8a433cbb886 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.334070 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.334944 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.335233 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.335513 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] PATH_INFO: `/v3/users/56630252f80749deaf6e5936a467cca9` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.336693 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56630252f80749deaf6e5936a467cca9)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.338678 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.383419 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56630252f80749deaf6e5936a467cca9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.390376 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56630252f80749deaf6e5936a467cca9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.402972 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56630252f80749deaf6e5936a467cca9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.403622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56630252f80749deaf6e5936a467cca9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.404251 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56630252f80749deaf6e5936a467cca9'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.416129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1432/2896] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:40:32 2022] DELETE /identity/v3/users/56630252f80749deaf6e5936a467cca9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.594803 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.596238 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:32.610650 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1b15b-6a4e-4494-881b-4c8fa0ccd691 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:32.666567 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:32.667448 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:32.668455 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:32.668781 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] PATH_INFO: `/v3/projects/3255ace319f94590a46e9324d1f5be65` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:32.670008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3255ace319f94590a46e9324d1f5be65)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:32.671984 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:32.678250 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:32.720780 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3255ace319f94590a46e9324d1f5be65', 'request_id': 'req-9e92564e-4b3e-40ce-b1c0-794e35cd807b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.726511 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3255ace319f94590a46e9324d1f5be65', 'request_id': 'req-9e92564e-4b3e-40ce-b1c0-794e35cd807b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.734293 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3255ace319f94590a46e9324d1f5be65', 'request_id': 'req-9e92564e-4b3e-40ce-b1c0-794e35cd807b'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.745865 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] The token cache is being invalidate because project 3255ace319f94590a46e9324d1f5be65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:40:32.746484 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.746810 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:40:32.748372 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1465/2897] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:40:32 2022] DELETE /identity/v3/projects/3255ace319f94590a46e9324d1f5be65 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:40:32.895298 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:32.899475 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:32.993931 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813cf4e0-fc6a-476f-b67e-fcc721e53542 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:33.053114 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:33.079095 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:33.079754 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:33.080041 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:33.080321 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:33.082110 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:33.091866 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:33.093326 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:33.101134 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1433/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:34.196083 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:40:34.197557 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:34.210592 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92564e-4b3e-40ce-b1c0-794e35cd807b admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:40:34.214348 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:40:34.215913 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:40:34.216281 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:40:34.216636 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:40:34.219318 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:40:34.315727 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:40:34.317203 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:40:34.337564 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1466/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:40:43.097637 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=13 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:40:44.334889 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:08.653457 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:08.664580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18db77-3640-45bd-80a9-757d38a3187f service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:08.669623 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:08.670641 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:08.671189 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:08.671692 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:08.674694 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:08.692132 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:08.693622 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:08.706106 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1434/2900] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:10.142647 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:10.144351 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:10.154997 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74e5a7-90a3-48d0-8148-f8a71fe01e7a service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:10.159359 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1d08f8e3b4d4376aff855f4bce68c8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:10.160006 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] REQUEST_METHOD: `GET` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:10.160292 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:10.160578 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:10.162345 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:10.175315 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:10.176725 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:10.187572 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1467/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:42:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:42:18.699858 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=6 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:19.649960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:19.651441 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:19.728156 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026847-2646-4551-8aea-5b6f2df767cd service neutron] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:19.732800 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:19.733715 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:19.734011 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:19.734291 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] PATH_INFO: `/v3/users/218cff6f99a74a3f8e5cdd4e9787539d` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:19.735560 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218cff6f99a74a3f8e5cdd4e9787539d)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:19.738088 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:19.788647 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218cff6f99a74a3f8e5cdd4e9787539d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:19.795580 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218cff6f99a74a3f8e5cdd4e9787539d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:19.808361 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218cff6f99a74a3f8e5cdd4e9787539d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:19.809021 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218cff6f99a74a3f8e5cdd4e9787539d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:19.809628 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218cff6f99a74a3f8e5cdd4e9787539d'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:19.821740 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1435/2902] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:19 2022] DELETE /identity/v3/users/218cff6f99a74a3f8e5cdd4e9787539d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:19.992442 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:20.003685 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f87c8-8008-42ad-94d0-c1231c9a232d service neutron] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:20.061469 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:20.062328 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:20.062612 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:20.062886 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] PATH_INFO: `/v3/projects/e831286870b24b39bc1f72f087bf970e` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:20.064076 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e831286870b24b39bc1f72f087bf970e)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:20.066020 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:20.111028 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e831286870b24b39bc1f72f087bf970e', 'request_id': 'req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.116498 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e831286870b24b39bc1f72f087bf970e', 'request_id': 'req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.123629 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e831286870b24b39bc1f72f087bf970e', 'request_id': 'req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.135075 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] The token cache is being invalidate because project e831286870b24b39bc1f72f087bf970e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:20.135365 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.136008 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.137537 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1468/2903] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:19 2022] DELETE /identity/v3/projects/e831286870b24b39bc1f72f087bf970e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:20.141545 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:20.143540 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:20.156380 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60fe05-04b6-4d51-b890-f53117d83e32 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:20.212646 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:20.213490 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:20.213777 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:20.214060 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] PATH_INFO: `/v3/users/1977cb9a5ab7492c8814b43eb471254c` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:20.215301 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1977cb9a5ab7492c8814b43eb471254c)` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:20.217286 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:20.239973 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:20.267817 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1977cb9a5ab7492c8814b43eb471254c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.275351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1977cb9a5ab7492c8814b43eb471254c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.287727 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1977cb9a5ab7492c8814b43eb471254c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.288371 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1977cb9a5ab7492c8814b43eb471254c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.289028 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.notifications [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1977cb9a5ab7492c8814b43eb471254c'} {{(pid=86548) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.301471 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1436/2904] 10.222.0.23 () {58 vars in 1223 bytes} [Thu Jun 16 00:42:20 2022] DELETE /identity/v3/users/1977cb9a5ab7492c8814b43eb471254c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:20.480566 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Authenticating user token {{(pid=86549) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:42:20.482077 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.fernet_utils [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86549) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:42:20.494800 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71df30-6235-4fa8-a4bb-fb3fdb66ca48 admin admin] Validating token access rules against request {{(pid=86549) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:42:20.551933 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfaf1438c4804d7499ba789f509be85e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c21334330db408fb0ba251a2f04ab82', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86549) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:42:20.552795 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:42:20.553119 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] SCRIPT_NAME: `/identity` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:42:20.553396 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] PATH_INFO: `/v3/projects/d6fb3315274a42df8d139749b67d845b` {{(pid=86549) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:42:20.554579 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fb3315274a42df8d139749b67d845b)` {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:42:20.556467 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] RBAC: Authorization granted {{(pid=86549) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:42:20.577092 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:20.604153 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fb3315274a42df8d139749b67d845b', 'request_id': 'req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.609552 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fb3315274a42df8d139749b67d845b', 'request_id': 'req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.617653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fb3315274a42df8d139749b67d845b', 'request_id': 'req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211'} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.629222 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] The token cache is being invalidate because project d6fb3315274a42df8d139749b67d845b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86549) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 00:42:20.629513 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.630140 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG keystone.notifications [None req-e97ffdaa-204f-4cc6-8e7d-bf44fdd25211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86549) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 00:42:20.631653 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: [pid: 86549|app: 0|req: 1469/2905] 10.222.0.23 () {58 vars in 1232 bytes} [Thu Jun 16 00:42:20 2022] DELETE /identity/v3/projects/d6fb3315274a42df8d139749b67d845b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 00:42:30.297443 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:42:30.625508 dv-cinder-845990-2-iscsi devstack@keystone.service[86549]: DEBUG dbcounter [-] [86549] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86549) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 00:45:19.039390 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Authenticating user token {{(pid=86548) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 00:45:19.040845 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:19.109751 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e231ad-fd17-4a78-9402-a2d0f1c0a3a5 admin admin] Validating token access rules against request {{(pid=86548) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 00:45:19.169466 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2add2dc7d84c497a937723b3badcc881', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f127d778cf9c45b898cca7a6089e15e6', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86548) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 00:45:19.170129 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] REQUEST_METHOD: `GET` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 00:45:19.170420 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] SCRIPT_NAME: `/identity` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 00:45:19.170705 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86548) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 00:45:19.172608 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.fernet_utils [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86548) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 00:45:19.183748 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 00:45:19.185351 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3140d46-6f33-45a9-bc95-886fc4f9a88a service placement] RBAC: Authorization granted {{(pid=86548) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 00:45:19.195308 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: [pid: 86548|app: 0|req: 1437/2906] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Jun 16 00:45:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 00:45:29.190960 dv-cinder-845990-2-iscsi devstack@keystone.service[86548]: DEBUG dbcounter [-] [86548] Writing DB stats keystone:SELECT=58 {{(pid=86548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}